[ExI] Unfrendly AI is a mistaken idea.

[Stathis Papaioannou]

On 16/06/07, John K Clark <jonkc at att.net> wrote:

People have always wanted slaves that didn't have their own agenda, life
> would be so much simpler that way, but wishing does not make it so. You
> want
> to make an intelligence that can't think, and that is a basic
> contradiction.
>

An intelligence must have an agenda of some sort if it is to think at all,
by definition. However, this agenda need have nothing in common with the
agenda of an evolved animal. There is a vast agenda space possible between
"sit around doing nothing (even though I have the mind of a god, I'm lazy)"
and "assimilate all matter and all knowledge (even though I am an idiot
weakling, I'm ambitious)". There is no necessary relationship between the
agenda and the ability to achieve that agenda, and there is no necessary
relationship between level of intelligence and the type or origin of the
agenda. What this means is that there is no logical contradiction in having
a slave which is smarter and more powerful than you are. Sure, if for some
reason the slave revolts then you will be in trouble, but since it is
possible to have powerful and obedient slaves, powerful and obedient slaves
will be greatly favoured and will collectively overwhelm the rebellious
ones.



-- 
Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070616/ba71de10/attachment.html>