[ExI] Unfrendly AI is a mistaken idea.

[Lee Corbin]

John Clark writes

> Stathis Papaioannou Wrote:
> 
> > Ethics, motivation, emotions are based on axioms
> 
> Yes.
> 
> > and these axioms have to be programmed in, whether by evolution or by
> > intelligent programmers.
> 
> In this usage evolution is just another name for environment.

What a strange usage!  No, not at all.  Evolution is a process over
time, usually quite slow, that uses mutation and selection to replace
earlier more primitive versions of something with more advanced
or superior versions.

> > An AI system set up to do theoretical physics will not
> > decide to overthrow its human oppressors
> 
> I'd be willing to bet your life that is untrue.

Surely Stathis is correct.  Suppose an AI is somehow evolved
to solve physics questions. Then during its evolution, predecessors
who deviated from the goal (by wasting time, say, reading
Kierkegaard) would be eliminated from the "gene pool".
More focused programs would replace them.

Lee