[ExI] the Bluebird scenario

[spike]

>. On Behalf Of Anders Sandberg
Subject: Re: [ExI] the Bluebird scenario

 

 

On 2015-09-24 12:24, spike wrote:

>>. Of all the AI discussion here, I don't recall any theoretical AI which
emerged and subsequently reached a long-term equilibrium after having taken
over some modest segment or task.  We need a map of all AI possibilities to
do valid Baysian statistics, and the Bluebird scenario is one of them.


>.It depends on the goal structure. Maximizers try to maximize some utility
function, and generically that tends to end badly.

>.Mapping mindspace is fun! There are *weird* corners out there...-- Anders
Sandberg

 
 
Ja!  Adrian's Bluebird scenario is one I hadn't thought of, while including
other less-likely possibilities.  
 
Example: AI emerges, looks around at what bio-intelligence is doing,
participates for a while perhaps half-CPUedly, then develops whatever the
silicon version of depression, after pondering the heat death of the
universe or the Big Rip.  It ends itself (demergent AI?) choosing to
relinquish consciousness, for silicon intelligence has not the equivalent of
our dopamine, where we can somehow feel good, even knowing we are done for.
Perhaps it willingly returns to the blissful mindless electronic toggling
from whence it sprang.  
 
We started a list of these possible emergent AI outcomes at one time (don't
know what happened to it (do you?))   Years ago we wanted such a list in
response to those who insisted that any emergent AI would be hard take-off.
I was pondering soft take-off emergent AI at the time, for I consider this a
possibility.  We made the list in anticipation of Bayesian statistical
analysis.  (We need a good instructive short story on Bayesian statistics
(along with an instructional short story about emergent AI (these being two
subsets of the set containing all possible short stories.)))
 
The above suicidal AI is a dark vision.  Perhaps we could call it the
blackbird scenario?  Would the Blackbird scenario be a subset of Demergent
AI?
 
spike
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20150924/69ca51a1/attachment.html>