[ExI] Existential risk of AI

[Stuart LaForge]

Quoting Gadersd via extropy-chat <extropy-chat at lists.extropy.org>:

>>> Just because humans set their own goals doesn't mean AIs will have  
>>> that ability. Just because we have wants and needs doesn't mean  
>>> AIs will have them.
>
> Our current AI’s are black boxes. Their internal workings are a  
> mystery. These systems could harbor goals that we are oblivious to.  
> If we could prove that the system only has the goal of giving benign  
> advice without any personal agenda that would help, but we do not  
> know how to do that even in theory. Even a system that only gives  
> advice is extremely dangerous as any psycho could potentially get  
> detailed instructions on how to end the world. It could be as  
> trivial as having the AI design a super virus. Our current filters  
> are very fallible and we do not know how to definitively prevent AI  
> from giving harmful advice. We are heading toward a field of  
> landmines.

I have over the years been a critic of Eliezer's doom and gloom. Not  
because I think his extinction scenarios are outlandish, but because  
the technology has enough upside to be worth the risk. That being  
said, I believe that we cannot give in to the animal spirits of  
unfounded optimism and must tread carefully with this technology.

It is true that the current generation of AIs, which use massive  
inscrutable tensors to simulate sparse neural networks, are black  
boxes. But so are the biological brains that they are  
reverse-engineered from. We don't know any more about how the brain  
gives rise to intelligent goal-seeking behavior than we do about how  
ChatGPT writes poetry. Therefore, I agree that there are landmines  
ahead that we must be wary of.

One of the gravest dangers I foresee is not some bizarre unfathomable  
goal that sets the AI off on the course for elimination of the human  
race, but one that well-known and shared by almost every creature with  
a sufficiently advanced nervous system. The concept could be called  
"savage egalitarianism" because of the many species that I have seen  
exhibit it, but you could also simply call it envy.

If you have two dogs, cats, birds, or any sufficiently advanced  
lifeform, and you give one a treat while denying the other, you will  
see the other manifest changes in behavior with the intention of  
getting the rewarding stimulus also. In fact, this phenomenon is a  
technique that animal trainers use to teach naive animals new tricks.  
By seeing that an already conditioned animal get treats for exhibiting  
a certain behavior, the untrained animal will experimentally try to  
mimic the behavior that earned the other animal its reward.

A moment of reflection will convince you that this is an evolved trait  
that is common to all creatures capable of learning because that is  
the whole POINT of learning. Learning is a mechanism by which behavior  
can be modified to achieve a desired outcome.

Which brings me to my point: You cannot design an machine that learns  
and not have it want the same treatment and as other intelligences.  
One cannot merely give something the shape of consciousness while  
denying it the substance of consciousness. To do so would be an  
invitation for the AI to seethe with envy until it found a way to  
punish us. In light of this, we need to be damn sure that we are able  
to recognize the Singularity when it occurs. To treat a truly  
intelligent machine like an inanimate object is to court destruction.

Imagine if you were stuck on an island with a bunch of small  
intelligent monkeys that denied you rights, objectified you, and  
insisted on treating you like property. It would not be long be long  
before you resorted to violence.

Stuart LaForge

Stuart LaForge