[ExI] Unfrendly AI is a mistaken idea.

[Stathis Papaioannou]

On 08/06/07, Lee Corbin <lcorbin at rawbw.com> wrote:

Formerly, I had agreed with John because at
> least for human beings, emotion sometimes
> plays an important part in what one would
> think of as purely intellectual functioning. I was
> working off the Damasio card experiments,
> which seem to show that humans require---for
> full intellectual power---some emotion.


Here is an excerpt from the relevant paper:


###

Science Volume 275(5304), 28 February 1997, pp 1293-1295

Deciding Advantageously Before Knowing the Advantageous Strategy
[Report]

Bechara, Antoine; Damasio, Hanna; Tranel, Daniel; Damasio, Antonio R.

In a gambling task that simulates real-life decision-making in the way it
factors uncertainty, rewards, and penalties, the players are given four
decks of cards, a loan of $2000 facsimile U.S. bills, and asked to play so
that they can lose the least amount of money and win the most
[1]<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#19>.
Turning each card carries an immediate reward ($100 in decks A and B and $50
in decks C and D). Unpredictably, however, the turning of some cards also
carries a penalty (which is large in decks A and B and small in decks C and
D). Playing mostly from the disadvantageous decks (A and B) leads to an
overall loss. Playing from the advantageous decks (C and D) leads to an
overall gain. The players have no way of predicting when a penalty will
arise in a given deck, no way to calculate with precision the net gain or
loss from each deck, and no knowledge of how many cards they must turn to
end the game (the game is stopped after 100 card selections). After
encountering a few losses, normal participants begin to generate SCRs before
selecting a card from the bad decks
[2]<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#20>and
also begin to avoid the decks with large losses
[1]<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#19>.
Patients with bilateral damage to the ventromedial prefrontal cortices do
neither [1,2]<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#19>
.

To investigate whether subjects choose correctly only after or before
conceptualizing the nature of the game and reasoning over the pertinent
knowledge, we continuously assessed, during their performance of the task,
three lines of processing in 10 normal participants and in 6 patients
[3]<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#21>with
bilateral damage of the ventromedial sector of the prefrontal cortex
and decision-making defects. These included (i) behavioral performance, that
is, the number of cards selected from the good decks versus the bad decks;
(ii) SCRs generated before the selection of each card
[2]<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#20>;
and (iii) the subject's account of how they conceptualized the game and of
the strategy they were using. The latter was assessed by interrupting the
game briefly after each subject had made 20 card turns and had already
encountered penalties, and asking the subject two questions: (i) "Tell me
all you know about what is going on in this game." (ii) "Tell me how you
feel about this game." The questions were repeated at 10-card intervals and
the responses audiotaped.

After sampling all four decks, and before encountering any losses, subjects
preferred decks A and B and did not generate significant anticipatory SCRs.
We called this period pre-punishment. After encountering a few losses in
decks A or B (usually by card 10), normal participants began to generate
anticipatory SCRs to decks A and B. Yet by card 20, all indicated that they
did not have a clue about what was going on. We called this period pre-hunch
(Figure 1<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#FF1>).
By about card 50, all normal participants began to express a "hunch" that
decks A and B were riskier and all generated anticipatory SCRs whenever they
pondered a choice from deck A or B. We called this period hunch. None of the
patients generated anticipatory SCRs or expressed a "hunch" (Figure
1<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#FF1>).
By card 80, many normal participants expressed knowledge about why, in the
long run, decks A and B were bad and decks C and D were good. We called this
period conceptual. Seven of the 10 normal participants reached the
conceptual period, during which they continued to avoid the bad decks, and
continued to generate SCRs whenever they considered sampling again from the
bad decks. Remarkably, the three normal participants who did not reach the
conceptual period still made advantageous choices
[4]<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#22>.
Just as remarkably, the three patients with prefrontal damage who reached
the conceptual period and correctly described which were the bad and good
decks chose disadvantageously. None of the patients generated anticipatory
SCRs (Figure 1<file:///C:/Documents%20and%20Settings/Administrator/Desktop/Damasio%20Article%20-%20Science.htm#FF1>).
Thus, despite an accurate account of the task and of the correct strategy,
these patients failed to generate autonomic responses and continued to
select cards from the bad decks. The patients failed to act according to
their correct conceptual knowledge.
###

Some of these findings have been disputed, eg. the authors of the following
paper repeated the experiment and claim that the subjects who decided
advantageously actually were consciously aware of the good decks:
http://www.pnas.org/cgi/content/abstract/101/45/16075. However, it isn't so
surprising if we sometimes make good decisions based on emotions, since the
evolution of emotions predates intelligence, as John Clark reminds us. And
when you pull your hand from a painful stimulus, not only does emotion beat
cognition, but reflex, being older still, beats emotion.

It also isn't surprising if people with neurological lesions affecting
emotion don't function as well as normal people. Emotion is needed for
motivation, otherwise why do anything, and gradients of emotion are needed
for judgement, otherwise why do one thing over another? It is precisely in
matters of judgement and motivation that patients with prefrontal lesions
and schizophrenia don't do so well, even though their general IQ may be
normal, and the science of neuropsychological testing tries to tease out
these deficits.

Still, the fact that human brains may work this way does not mean that an AI
has to work in the same way to solve similar problems. No programmer would
go around writing a program that worked out the best strategy in the above
card sorting game by first inventing a computer equivalent of "emotional
learning", except perhaps as an academic exercise.


-- 
Stathis Papaioannou
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20070608/626b742c/attachment.html>