[extropy-chat] Re: Structure of AI

[Eliezer Yudkowsky]

Adrian Tymes wrote:
> --- Eliezer Yudkowsky <sentience at pobox.com> wrote:
> 
>> It may not sound practically important (how many maxentropy universes
>> did we plan to run into, anyway?) but from a theoretical standpoint
>> this is one hell of a huge nitpick:  The real universe is an atypical
>> special case.
> 
> It's also the only one that matters.  Any and all efforts to deal with
> universes radically different than the one we actually face are wasted.

I wasn't previously planning to work on that, but now that you mention it, 
it might be a good way to stress-test the basic concepts, for the same 
reason that people run really weird HTML through their browsers to see if 
they crash.  How do you get an exact analysis of which universe you live 
in?  Human minds can imagine alternate possibilities, and this is a fine 
talent to have, especially if you're not sure which possibility is real.

> That this type of analysis is even considered in the SIAI's effort to
> build FAI leads me to conclude that the SIAI is not worth funding, even
> if FAI itself would be a desirable goal. 

Now you're just being silly.  Don't tell me what I may or may not imagine 
to kick-start my thinking.

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence