<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Forwarded message:<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>From: Omar Rahman<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Original message follows:<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span class=apple-style-span><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif"'>From: Anders Sandberg <<a href="mailto:anders@aleph.se">anders@aleph.se</a>></span></span><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif"'><br><br></span><o:p></o:p></p><p class=MsoNormal><br><br><o:p></o:p></p><p class=MsoNormal><span class=apple-style-span><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif"'>One of the things that struck me during our Winter Intelligence workshop </span></span><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif"'><br><span class=apple-style-span>on intelligence explosions was how confident some people were about the </span><br><span class=apple-style-span>speed of recursive self-improvement of AIs, brain emulation collectivies </span><br><span class=apple-style-span>or economies. Some thought it was going to be fast in comparision to </span><br><span class=apple-style-span>societal adaptation and development timescales (creating a winner takes </span><br><span class=apple-style-span>all situation), some thought it would be slow enough for multiple </span><br><span class=apple-style-span>superintelligent agents to emerge. This issue is at the root of many key </span><br><span class=apple-style-span>questions about the singularity (one superintelligence or many? how much </span><br><span class=apple-style-span>does friendliness matter?)</span><br><br><span class=apple-style-span>It would be interesting to hear this list's take on it: what do you </span><br><span class=apple-style-span>think is the key limiting factor for how fast intelligence can amplify </span><br><span class=apple-style-span>itself?</span><br><br><span class=apple-style-span>Some factors that have been mentioned in past discussions:</span><br><span class=apple-style-span> Economic growth rate</span><br><span class=apple-style-span> Investment availability</span><br><span class=apple-style-span> Gathering of empirical information (experimentation, interacting </span><br><span class=apple-style-span>with an environment)</span><br><span class=apple-style-span> Software complexity</span><br><br></span><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>'Software complexity' stands out to me as the big limiting factor. Assuming it applies at all to machine intelligences, Godel's incompleteness theorem would seem to imply that once this thing starts off it can't just go forward with some 'ever greater intelligence' recipe. Add to this the cognitive load of managing a larger and larger system and the system will have to optimize and, oddly enough, 'automate' subprocesses; much as we don't consciously breath, but we can control it if we wish.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Once this thing hits it's 'Godel Limit' if it wishes to progress further it will be forced into the 'gathering of empirical information', at this stage it is unknown how long it will take for a new axiom to be discovered.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span class=apple-style-span><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif"'> Hardware demands vs. available hardvare</span></span><span style='font-size:13.5pt;font-family:"Helvetica","sans-serif"'><br><span class=apple-style-span> Bandwidth</span><br><span class=apple-style-span> Lightspeed lags</span><br><br><span class=apple-style-span>Clearly many more can be suggested. But which bottlenecks are the most </span><br><span class=apple-style-span>limiting, and how can this be ascertained?</span><br><br><br></span><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Many people seem to assume that greater intelligence is simple a matter of 'horsepower' or more processing units or what have you. My analogy would be a distribution network where you get more and more carrying capacity as you add more trucks but once you reach the ocean adding more trucks won't help. Unless you fill the ocean with trucks I suppose. =D<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Does anyone care to address my main assumption? Does Godel's incompleteness theorem apply?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Regards,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Omar Rahman<o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><o:p> </o:p></span></p></div></body></html>