<html><head></head><body><div><span data-mailaddress="dennislmay@yahoo.com" data-contactname="Dennis May" class="clickable"><span title="dennislmay@yahoo.com">Dennis May</span><span class="detail"> <dennislmay@yahoo.com></span></span> , 14/6/2014 9:13 PM:<br><blockquote class="mori" style="margin:0 0 0 .8ex;border-left:2px blue solid;padding-left:1ex;"><div style="color:#000;background-color:#fff;font-family:garamond, new york, times, serif;font-size:18pt;"><div><span style="font-family: 'Times New Roman', serif; font-size: 14pt;">Questions
concerning the Fermi Paradox should include the variable of</span></div><div style="margin:0in 0in 0pt;"><span style="font-family:"Times New Roman","serif";font-size:14pt;">adversaries
at every juncture since all of biology is known to deal with </span></div><div style="margin:0in 0in 0pt;"><span style="font-family:"Times New Roman","serif";font-size:14pt;">the
issue continually from the earliest systems forward.</span></div></div></blockquote></div><div><br></div><div>This is a problematic approach. Yes, freely evolving systems of replicators generically get parasitism. But in the Fermi context free evolution is just one option: a civilization that has developed into a singleton might coordinate future behaviour to preclude parasitism or adversarial behaviour, or it might decide on "intelligent design" of its future development. If it is also alone within the reachable volume its dynamics will be entirely adversary-free. Maybe this is not the most likely case, but it it has to be analysed - and understanding it is pretty essential for being able to ground the adversarial cases. </div><div><br></div><div>When Joanna Bryson gave a talk here ( it can be viewed at https://www.youtube.com/watch?v=wtxoNap_UBc ) she also used a biological/evolutionary argument for why we do not need to worry about the AI part of the intelligence explosion; as I argued during the Q&A, there might be a problem in relying too much on biological insights when speaking about complex agent systems. Economics, another discipline of complex systems, gives very different intuitions. </div><div><br></div><div>Then again, I do think running game theory for Fermi is a good idea. I had a poster about it last summer: https://dl.dropboxusercontent.com/u/50947659/huntersinthedark.pdf In this case I think I showed that some berserker scenarios are unstable. (And thanks to Robin for posing the issue like this - we ought to write the paper soon :-) )</div><div><br></div><br><div><blockquote class="mori" style="margin:0 0 0 .8ex;border-left:2px blue solid;padding-left:1ex;"><div style="color:#000;background-color:#fff;font-family:garamond, new york, times, serif;font-size:18pt;"><div style="margin:0in 0in 0pt;"><span style="font-family:"Times New Roman","serif";font-size:14pt;"></span></div><div><span style="font-family: 'Times New Roman', serif; font-size: 14pt;">Once super-intelligences are in competition</span></div><div style="margin:0in 0in 0pt;"><span style="font-family:"Times New Roman","serif";font-size:14pt;">I
would expect things to get very complicated concerning the continued</span></div><div style="margin:0in 0in 0pt;"><span style="font-family:"Times New Roman","serif";font-size:14pt;">advantage
of “size” versus many other variables becoming enabled.</span></div></div></blockquote></div><div><br></div>We know that game theory between agents modelling each other can easily become NP-complete (or co-NP):<div>https://www.sciencedirect.com/science/article/pii/S0004370206000397?np=y</div><div>https://www.sciencedirect.com/science/article/pii/S0004370297000301?np=y</div><div>And these are bounded agents; superintelligences will create an even more complex situation.</div><div><br></div><div>Of course, as seen in this little essay,</div><div>http://www.aleph.se/andart/archives/2009/10/how_to_survive_among_unfriendly_superintelligences.html</div><div>non-superintelligences can thrive under some circumstances simply because they go under the radar. A bit like how many insects do not use adaptive immune systems, or certain military devices are not armoured - it is not worth increasing resilience of individuals when you can get resilience by having many instances. <br><div></div><br><br>Anders Sandberg, Future of Humanity Institute Philosophy Faculty of Oxford University</div></body></html>