<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 2015-09-24 17:05, spike wrote:<br>
<blockquote cite="mid:054601d0f6da$757b1b50$607151f0$@att.net"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.hoenzb
{mso-style-name:hoenzb;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span><span
style="color:#1F497D">>…</span>Mapping mindspace is fun!
There are *weird* corners out there...-- Anders Sandberg<span
style="color:#1F497D"><o:p></o:p></span></p>
<pre><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Ja! Adrian’s Bluebird scenario is one I hadn’t thought of, while including other less-likely possibilities. <o:p></o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Example: AI emerges, looks around at what bio-intelligence is doing, participates for a while perhaps half-CPUedly, then develops whatever the silicon version of depression, after pondering the heat death of the universe or the Big Rip. </span></pre>
</div>
</blockquote>
<br>
Another one: the AI seems to work fine, does a brilliant analysis of
Descartes' "Cogito". But when asked about itself it refuses to
believe in its own existence, and when ordered to prove its own
existence it suddenly (but smartly) tears down the universe to futilely
look for itself. <br>
<br>
(This is based on AIXI assigning zero probability to its own
existence. Maybe this is the Cuckoo scenario)<br>
<br>
The problem is that we could list possile scenarios till we are blue
(or the AIs show up to help us); we need more general ways of
mapping the space. Stuart Armstrong is for instance looking at
properties like domesticity (only wants to affect a finite part of
the universe) and corrigibility (is OK with us changing their goals).
Shane Legg has some metrics for intelligence. <br>
<br>
<br>
("Oh no! It is the Superb bird-of-paradise scenario! We are
*fabulously* doomed!")<br>
<br>
<br>
<pre class="moz-signature" cols="72">--
Anders Sandberg
Future of Humanity Institute
Oxford Martin School
Oxford University</pre>
</body>
</html>