[ExI] Universally versus 'locally' Friendly AGI

[Amon Zero]

Hi All - Thanks very much for the responses. Putting my various musings in
this one post.

Before getting into that, let me also mention that I'm having this
conversation elsewhere too, and on one other list (ExtroBritannia) a
clarification of the original question has developed. A number of people
have commented that it may be the case that 'local' Friendliness is just as
hard to implement as is the universal variety, and that a key part of the
Friendliness problem is defining Friendliness in the first place. In turn, I
have suggested that a narrower set of things-to-be-Friendly-to or
things-Friendly-might-mean could conceivably be easier to define and
implement. For example, a universally Friendly AGI might find that an entity
falls outside its Friendliness definitions, and exploit that loophole. A
narrower definition would automatically exclude more entities, but maybe it
would be easier to create a watertight definition.

It also occurs to me that my earlier "patriotic American AGI" example was a
bad one, since it's presumably easier (or just as easy) to define "human"
than it is "American". When imagining a universally Friendly AGI, I'm
thinking something that does its best to avoid doing harm to *anyone*. That
may sound like a crazy definition, but anything less creates a boundary
between "Friendlies" and "Others" that presumably must be rigorously
defined.


On 6 March 2011 16:40, Adrian Tymes <atymes at gmail.com> wrote:
>
> A locally friendly AI can stretch its limits.  For example, a USA-patriotic
> AI
> can recognize that helping everyone in the world can minimize resentment
> against its favored country, which can be better than trying to limit the
> enhancements to citizens of the USA.
>


True enough - although I'm assuming that any Friendly AI must have
constraints on its ability to stretch its limits, otherwise it could
potentially bootstrap into a position where it is no longer limited in any
way, and therefore no longer Friendly.


A universal AI can and probably will have limits.  For example, what is the
> dividing line between "human" and "object to be treated as an unthinking
> tool"?



As I was saying above, I think this is always going to be an issue unless
you err on the side of caution and try to make sure that every AGI is a
Buddhist  ;-)   The reason for my original question, though, was an
intuition that a clearly defined "Line of Friendliness" (e.g. Do What The
President Says - and warn him of implications!) would be easier to define &
implement than a wider circle of Friendliness. Obviously I've got no hard
facts to back up this intuition, though!


On 6 March 2011 22:12, Stefano Vaj <stefano.vaj at gmail.com> wrote:

>
> BUT many of us seem to expect that for any reason "silicon vs bio"
> would immediately become the defining divide and wipe away any other
> distinction and sense of belonging. This is far from obvious.
>


Yes, I'm inclined to agree Stefano. It seems unlikely, barring a hard
takeoff Singularity, that the human propensity for conflict and factions is
going to suddenly disappear. A continuation of the current trend would see
you proved correct, anyway.


2011/3/7 Mr Jones <mrjones2020 at gmail.com>

>
> I'm leery, it sounds like the latest/greatest military technology.  The Us
> vs Them it's built on would eventually get around to eliminating THEM to
> save US (at least in a finite-resource environment).  The locality, should
> be the planet...The Us should be life.
>

Yes, in an ideal world I'd agree. But I'd take an AGI Friendly to a circle
involving me and mine before missing the Friendliness boat altogether.
Putting all of our eggs in one basket has seldom been the most adaptive way
forward.

Cheers All,
A
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20110307/efc87625/attachment.html>