[ExI] Unfrendly AI is a mistaken idea.

[Eugen Leitl]

On Wed, Jun 06, 2007 at 09:33:17PM +1000, Stathis Papaioannou wrote:

>    It's valuable in understanding how human minds work, but when you turn
>    it to other matters it leads to religion.

Iterated evolutionary interactions require the capability to model
self and opponents, which implies ability to deceive and detect
deceit. Any system operating in the marketplace needs to be able
to do that, for instance.

Most things people call anthropomorphic is based on a simplistic
human model (many programmers are guilty of that). There are some
frozen randomness, a lot of these things people think are bugs and
warts are features, and critical features.

People keep misunderestimating people.