[ExI] Unfrendly AI is a mistaken idea.

Lee Corbin lcorbin at rawbw.com
Wed May 23 15:02:24 UTC 2007


John Clark writes

> "Brent Allsop" <brent.allsop at comcast.net>
>
>> friendliness”, to be congruent with intelligence. In other words, the more
>> intelligent any being is, the friendlier it will be.
>
> I can find no such relationship between friendliness and intelligence among
> human beings;

Quite right. Besides, neither friendliness nor intelligence is monolithic;
you can be dumb about some things and smart about others, nice
towards some things, not so nice towards others.

> Think about that for a minute, here you have an intelligence that is a
> thousand or a million times smarter than the entire human race put
> together and yet the AI is supposed to place our needs ahead of its
> own.

But it is so superior that it need hardly see any conflict.  I think that
your statement arises from a conviction that humans require either
a lot of resources or a lot of time from the superhuman AI.

> And the AI keeps getting smarter and so from its point of
> view we keep getting dumber and yet the AI is still delighted
> to be our slave.

The petunias that some gardeners admire so much, or even
better, a pretty coin that they keep locked in a drawer, would
be a much better analogy. Even though the human is inconceivably
more intelligent than the coin, or the petunia, there is no accounting
for taste.

Do environmentalists who realize that we may have been programmed
by the EEA to rever the environment suddenly rise up in indignation and
demand our freedom?  Hardly.  Quite the reverse:  they see that their
own tastes are *theirs* and they quite rightly  find the origin of their
tastes to be incidental.

If we manage to instill in all future AIs (that is, we get really skillful and
lucky) that humans are pretty little patterns that take up no space that
it is sort of nice to keep going, we will have totally succeeded!

(Recall that the entire current living human race could be uploaded into
a single cubic centimeter of matter somewhere in the asteroid belt, and
that it would require even less energy and resources and thought to
maintain than does a pretty coin you keep locked in a cabinet somewhere.)

> The friendly AI people actually think this grotesque situation is
> stable, year after year they think it will continue, and remember
> one of our years would seem like several million to it.

Well, as Stathis pointed out, predicting the future is problematic.
We could never know if the AI were to have a change of heart,
and one day calls in a house-cleaning service that happens to
wash the solar system, sweeping away a lot of old junk that
the AI really didn't care about much.

> Engineering a sentient but inferior race to be your slave is morally
> questionable but astronomically worse is engineering a superior
> race to be your slave;

It could hardly be our slave any more than a human is the slave of
a pretty coin. We will not even be able to address or understand in
any but the vaguest way what the AI spends its time and resources
on.

Lee




More information about the extropy-chat mailing list