[ExI] Unfrendly AI is a mistaken idea.

[Lee Corbin]

I had written

>> Regarding an extremely advanced AI, we can hope that the human
>> race becomes to it  as an old photograph is to us, one merely kept
>> around for sentimental reasons.
>> 
>> For just as you might keep a photograph of your great great grandfather,
>> so might an AI (friendly or not) wish to maintain an infinitesimal record of
>> its progenitors.

To readers of the original post, I hope that you understood that 
my analogy intentionally omitted one important aspect, namely
that a "still" photograph cannot in any way be said to be obtaining
benefit, whereas we want some at least minimal amount of runtime
for humanity and for ourselves. I had also mentioned a gardiner's
affection for his petunias, but discarded that solely for the reason
that an AI that takes over the Solar System is in all likelihood 
going to be so VASTLY more advanced than we are  that the
analogy of us to petunias is incorrect.

Richard writes

> But this presupposes something that I do not think will happen:  that 
> there will be a clear dividing line between us and these extremely 
> advanced AIs, in the same way that there is currently a clear dividing 
> line between us and our pets.

Of course there are many cases; evidently we assign somewhat different
probabilities.  

> I have never subscribed to this idea:  it has always seemed a given that 
> we will be able to move fluidly between our level of intelligence and 
> that of these 'higher' creatures.
> 
> Likewise, I think they will be just as interested in pouring their 
> consciousness back and forth between different vessels.  It will 
> probably be one of the main activities of both ourselves and these 
> superintelligences, to explore different sensoria, different powers of 
> thought, different types of consciousness.  Who wouldn't want to be a 
> cat for a day?

I cannot imagine that I will at any time prefer a handicapped intelligence.
Your idea strikes me as really weird. You want to waste a day being a
cat?

Now remember that even anything as primitive as an orginal human
being (uploaded, no doubt) will still have total formal control over
his emotions. I guess you want to be a happy cat. Well, why not be
an even happier advanced human and be able to appreciate it more?

> Who wouldn't occasionally want the simplicity of being 
> humble gardener, free from all this knowledge about the universe that 
> can sometimes get to be a burden?

John Clark and me.

Lee