[ExI] Unfriendly AI is a mistaken idea

[Lee Corbin]

Samantha writes

> To think that a machine that is a glorified theorem prover is
> going to spontaneously extrapolate all the ways it might get
> better and spontaneously sprout perfect general concept
> extraction and learning algorithms and spontaneously come
> to understand software and hardware in depth and develop
> a will to be better greater than all other consideration
> whatsoever but without ever in its self-improvement
> questioning that will/goal and somehow get or be given
> all the resources needed to eventually convert everything
> to highly efficient computational matrix is utterly and
> completely bizarre when you think about it.

I totally agree with your words "spontaneously extrapolate
all the ways it might get better".  It's not going to do that
unless it's designed to do that by us and by previous
versions of itself, OR---and this is the big "or"---these
talents evolve spontaneously in a highly competitive
environment where over some lengthy period of time
multiple AIs and their descendants compete vigorously
against each other.

And, naturally, what seems a long time to them might
pass pretty quickly for us. 

Lee