[ExI] far future

[Anders Sandberg]

On 09/01/2014 21:39, Ben wrote:
> It occurs to me that minds that were evolved to cope with the African
> Savannah on approximately the metre scale, are woefully inequipped for
> understanding how the universe works.  The reason that the double-slit
> experiment, and all the other manifestations of quantum theory (and many
> other things) are mind-blowing to us is probably that our minds never
> needed to understand those things during our evolutionary development.
> That kind of understanding will probably require a different mental
> architecture than the one we have.  It's likely that anyone capable of
> understanding how the universe works can't be 'human' in any
> currently-accepted sense.

I think this is right.

Our perceptual and conceptual apparatus is demonstrably weak when moving 
out of our domains of evolutionary adaptiveness. Just look at how hard 
it is to think about implications in logic correctly, let alone 
multidimensional spaces. The fact that it can handle so much is amazing, 
and might even hint at some kind of "Turing universality" of our 
thinking. But even if things can in principle be thought or understood 
by human-style minds they might be too slow or big to be thought in 
practice.

An even more worrying possibility might be that our motivation systems 
(and hence emotions and ethics) are also wrong for the universe at large.

In the most general case the No Free Lunch Theorems kindly tell us not 
to bother: no mind is better than any other mind across the space of all 
possible universes. But this global egalitarian nihilism doesn't apply 
to our universe, since it is a special case: not everything is equally 
likely, and for any utility function there exist more or less adequate 
minds when trying to maximize it in our universe.

The moral is not to trust intuitions outside our domains of evolutionary 
adaptiveness. And to embrace the individuality of this particular 
universe with all its quirks. It got just one time dimension, scandium, 
maybe Thorne-Żytkow objects, and power-law distributions everywhere! Yay!


-- 
Anders Sandberg,
Future of Humanity Institute
Oxford Martin School
Faculty of Philosophy
Oxford University