[ExI] AI motivation, was malevolent machines

[Rafal Smigrodzki]

On Sat, Apr 12, 2014 at 4:48 PM, Keith Henson <hkeithhenson at gmail.com>wrote:
>
>
> An interesting point that might help understanding is *why* we are
> mostly not conscious of our motives.  Even if I am aware that I must
> have this motivation for status seeking, it's an abstract intellectual
> awareness, not a reason to get up in the morning.  There must be some
> reproductive success element in not being aware of our own
> motivations.  Perhaps we need to hide them even from the rest of our
> minds to keep them from being too obvious to other social primates.
>
> ### Self-awareness of the type you mention is a neurological function. As
such, for it to evolve, there must be genes directing biological events,
and usage of metabolic resources for it to function. But, if self-awareness
does not increase fitness, genes for it will not be selected, and if it
does sometimes appear, it will be selected against to conserve energy.
Generally, unless it's evolutionarily useful or a side-effect of something
useful, it doesn't evolve, and if it does, it does not stay long, whatever
it is.

I don't believe in the self-deception explanation, the idea that our true
wicked self must be kept hidden from us to better lie to others. The truth
does not *have* to be hidden, it just does not have a reason to be known to
us. There is lack of selection for self-awareness about many levels of our
motivations, rather than active selection against it.

Rafal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20140413/fcb37832/attachment.html>