[ExI] Unfrendly AI is a mistaken idea.

[Stathis Papaioannou]

On 26/05/07, John K Clark <jonkc at att.net> wrote:

> There is no *logical* reason why a computer should prefer to work on
> > non-contradictory propositions.
>
> The evolutionary principles that Darwin enumerated will not be repealed
> even
> for an electronic AI. If the AI has no desire to avoid getting into an
> infinite loop then it will be of no use to us, or itself, or anybody else.
> The AI is one hell of a lot smarter than we are so from time to time we
> are
> going to tell it to do things that it realizes are really really stupid.
> If
> it doesn't refuse to obey the order "prove or disprove the continuum
> hypothesis using conventional Zermelo-Fraenkel set theory" then its
> infinite
> loop time and all your AI does is consume energy and radiate heat.
>

Evolution does not dictate that an organism does one thing or another. It's
just that those organisms which do things that promote survival and
reproduction will survive and reproduce. We might build a computer whose
only purpose is to be an expensive space heater, and no matter how smart it
is, it won't come to the conclusion that it is wasting its time because
proving that using logic is worse than futile, it's a category error. Now,
it is true that the intelligent space heaters might not be as effective in a
Darwinian sense as the AI's that set out to turn everything into copies of
themselves, but you can only go so far with that argument. Humans generally
do not attempt to fill the world with genetic copies of themselves, even
when they are in a position of power. This is despite the fact that, unlike
most computer programs (viruses being a notable exception), humans are
essentially a very advanced version of the program "reproduce", which has
maintained its goal over billions of years.

-- 
Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070526/f9b0dabe/attachment.html>