[ExI] singularity talk at lmco

Anders Sandberg anders at aleph.se
Thu May 14 10:02:21 UTC 2015

spike <spike66 at att.net> , 14/5/2015 1:16 AM:

I had an idea for bringing about the singularity.  We think of it as a program which (somehow) develops human intelligence and motives, but we as a species have demonstrated we have no clue how to create that software.  Suppose we theorize that a singularity could arise from a gigabyte program (assuming away comments and any embedded documentation, spaces, visual breaks etc) so that’s 10^9 bytes or 10^12 bits.  We start with 10^12 bits, set them all to zero, run that, if no singularity, set the first bit to 1, run that, no singularity, second bit to 1 first to 0, and so on.  Keep incrementing thru all 10^2 bits and hope that the first (in numerical order) intelligent program is friendly.  Then if no singularity occurs, we know that no gigabyte program can become intelligent and end humanity and all life as we know it. 

There is of course the obvious problem that 2^2^12 runs is a *tad* more than what can be done in the universe. But there is also another cool problem: the time needed to check whether you get a singularity. 

Right now AIXI(tl) is actually running on some computers and actually building towards a singularity - one can prove that it will eventually become as smart as (or smarter than) everything else. Except that the code is doubly exponential in time. Protons will have decayed before it will come up with any idea. 

Then there is the Busy Beaver problem: we know even very short programs can produce very long - but eventually halting - executions. The upper bound (Rado's sigma function) grows *very* fast with program length, in fact so fast that it is uncomputable using Turing machines! But this also means we can get "fake singularities" that run amazingly for a very long while and then just stop. 

So on one hand you could have real singularities like AIXI(tl) that are immensely slow, and fake ones that actually aren't going indefinitely but you cannot wait long enough to see that they are fake. In fact, by the uncomputability of the sigma function, I think it follows that there is no Turing-program that can tell them apart even in theory!

Anders Sandberg, Future of Humanity Institute Philosophy Faculty of Oxford University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20150514/2cb932f1/attachment.html>

More information about the extropy-chat mailing list