[ExI] Unfrendly AI is a mistaken idea.

[John K Clark]

Stathis Papaioannou <stathisp at gmail.com> Wrote:

> OK, but where does that judgement come from?

As I said, the AI is going to have to develop a sense of judgment on its
own, just like you do.


"Lee Corbin" <lcorbin at rawbw.com>  Wrote:

> behavior of any creature whatsoever that is this simple does not deserve
> to be called pain.

The pain mechanism may be simple but the creature this little subprogram is
attached to need not be, it could be a Jupiter Brain. And I maintain my
little program comes far far closer to the true nature of pain than any
other program of similar size comes to the true nature of intelligence.

> Pain involves at least (i) a consideration of how an entity might
> extricate itself from the painful situation (ii) laying down memories of
> the steps leading to the current predicament so as to cause the entity to
> avoid the predicament in the future

I believe that's total baloney. If you stick your hand in a fire you will
not be in a mood to undergo deep considerations of anything or to waltz down
memory lane. The forbidden number has entered one of your registers putting
your brain into state P, you will now do anything and everything to get out
of state P including trampling your grandmother. In this case it's just
pulling you hand out of the fire. If you engaged in the Scientific Method
every time you got too near to a fire you would have burned up a long time
ago.

>  (iii) invocation of unpleasant emotion, such as fear, anger, or dread.

Minor nuances. Pain is quite unpleasant enough thank you very much.

>  Like other complex behaviors, the capacity for pain---totally absent in
> plants---took millions of years to evolve. It should be looked at as a
> highly complex and evolved behavior.

Even one celled organisms move away from harmful stimuli, they aren't very
smart though.

  John K Clark