[ExI] Eliezer S. Yudkowsky, Singularitarian Principles. Update?

Aleksei Riikonen aleksei at iki.fi
Tue Nov 16 12:26:08 UTC 2010


On Tue, Nov 16, 2010 at 8:24 AM, Samantha Atkins <sjatkins at mac.com> wrote:
>
> That is not what I want.  I want to know what the current working theories are
> concerning FAI and and what type of FAI is the current working plan, if any.
> For a time it seemed to be CEV.  But some people in SIAI claim that is
> obsolete while others say it is still the general plan.  So I would like clarification.

The CEV page was published over 6 years ago, and already *two days*
after it was published an update was put out that actually, CEV
doesn't work as a specification of Friendliness. You can see that
clarification appended to the top of the CEV page.

To put it simply, SIAI currently *doesn't know* how to build FAI.
They're trying to solve open problems in mathematics (decision theory)
that need to be solved before a FAI specification would be possible.

(And personally, I expect that SIAI will eventually classify those
problems as so difficult that the primary plan should be to try to
navigate a Singularity *without* a solution to FAI.)


>> http://singinst.org/riskintro/index.html
>
> It is a start but not sufficient.  It doesn't really propose much of anything.

It proposes e.g. large new research disciplines within some fields of
science. Bigger things than a single institution would be capable of
on it's own.

What you seem to be asking for is a proposed solution to FAI, and not
accepting the answer that SIAI currently doesn't have a solution.
Similarly, as for much of the time the Manhattan Project was in
existence, they still couldn't tell how to build a nuke. They had to
do the actual research first. Only then can you draw up a
specification, and build what the specification says.

-- 
Aleksei Riikonen - http://www.iki.fi/aleksei




More information about the extropy-chat mailing list