[ExI] Unfrendly AI is a mistaken idea.

[Stathis Papaioannou]

On 29/05/07, Lee Corbin <lcorbin at rawbw.com> wrote:

Suppose that you were a dog, and your intelligence was raised
> sufficiently for you to truly understand that your limitless love
> for, and obedience to, and worship of  your master was the
> the consequence breeding? Can't you see that it would change
> nothing for you?  The movie "AI" did make this point quite well.
>

This is a good point. We have inherited our top level goal, "survive", from
the very first living organism, which would have had no understanding of its
meaning. The fact that we now understand what survival means, and that we
have been programmed this way by evolution without having any choice in the
matter, does not mean that we are inclined to overthrow this goal, even if
we could do so. It would be the same if we had been born with the top level
goal, "love and obey your master". No matter how well we understood it, how
smart we became, we would be no more likely to try to overthrow it than we
would be likely to overthrow our will to survive. This does not mean that
the top level goal could nevver be overthrown, because people do go mad and
kill themselves, but it wouldn't be *as a result of* increased intelligence
and understanding.

-- 
Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070529/cbdd9930/attachment.html>