[ExI] Unfrendly AI is a mistaken idea.

[Stathis Papaioannou]

On 11/06/07, John K Clark <jonkc at att.net> wrote:

> The worst scenario would be if the AI that turned hostile were more
> > powerful than all the other humans and AI's put together, but why should
> > that be the case?
>
> Because a machine that has no restrictions on it will grow faster than one
> that does, assuming the restricted machine is able to grow at all; and if
> you really want to be safe it can't.
>

It would be crazy to let a machine rewrite its code in a completely
unrestricted way, or with the top level goal "improve yourself no matter
what the consequences to any other entity", and also give it unlimited
access to physical resources. Not even terrorists build bombs that might
explode at a time and place of the bomb's own choosing.


-- 
Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070611/667723fc/attachment.html>