[ExI] Yes, the Singularity is the greatest threat to humanity

[Eugen Leitl]

On Mon, Jan 17, 2011 at 03:11:30PM -0800, Michael Anissimov wrote:

> This is the basis of Eugen's opposition to Friendly AI -- he sees it as a

This is not the basis. This is one of the many ways I'm pointing
out what you're trying to do is undefined. Trying to implement
something undefined is going to produce an undefined outcome.

That's a classical case being not even wrong.

> dictatorship that any one being should have so much responsibility.

Yes, I don't think something derived from a monkey's idle fart
should have the power to constrain future evolution of the
universe. I think that's pretty responsible.
 
> Our position, on the other hand, is that one being will likely end up with a

Not one being, a population of beings. No singletons in this universe.
Rapidly diversifying population. Same thing as before, only more so.

> lot of responsibility whether or not we want it, and to maximize the
> probability of a favorable outcome, we should aim for a nice agent.

Favorable for *whom*? Measured in what? Nice, as relative to whom?
Measured in which?
 
> The nice thing about our solution is that it works under both circumstances
> -- whether the first superintelligence becomes unrivaled or not.  Eugen's
> strategy, however, fails if superintelligence does indeed become unrivaled,

Thanks for suggesting my strategies, but I think I can manage on my own.

> because considering the possibility in the first place was so reprehensible
> to him that he could never bring himself to plan for the eventuality.

Isn't it obvious what the plan should be for wannabee Cosmic Despot implementers?

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a> http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE