[ExI] Singletons

Eugen Leitl eugen at leitl.org
Sat Jan 1 16:42:11 UTC 2011


On Sat, Jan 01, 2011 at 12:23:59PM +0100, Anders Sandberg wrote:

> A singleton doesn't necessarily have to be synchronized. Imagine a set  
> of local rules that gets replicated, keeping things constrained wherever  
> they go.

The problem that the local rules cannot be static, if the underlying
substrate isn't. And if there's life, it's not static. Unless the
cop keeps beating you into submission every time you deviate from
the rules. 

And if the rule set evolution is open ended, it is uncomputable.
So why having a baton-twirling cop on each block in the first place?
Some of them will wind up abusive, or crooked.

> I have the same aversion. However, I am open to the possibility that a  
> civilization without global coordination that really can put its foot  
> down and say "no!" to some activities will with a high probability be  
> wiped out by some xrisk or misevolution. I am still not convinced that  

That will only happen locally. The problem of invasive species is
solved that the ecosystem enters a new equilibrium. 

> this possibility is the truth, but it seems about as likely as the  
> opposite case.
>
> I would love to be able to find some good arguments that settle things  
> one way or another. The problem is that the xrisk category is pretty big  
> and messy, with unknown unknowns.

Let's look at a population of cultures the size of a galaxy. How do
you produce an existential risk within a single system that can wipe 
more than a stellar system? In order to produce larger scale mayhem
you need to utilize the resources of a large number of stellar
systems concertedly, which requires large scale cooperation of
pangalactic EvilDoers(tm).

> Military budgets are a few percent of GDP for heavily armed countries,  
> and maybe equally large for policing. In our bodies the immune system  
> accounts for ~20% of metabolism if I remember right.

The immune system is a good example of an evolved system that is guarding
self from non-self, with the energy and false positives tradeoff.

The problem of the hypothetical cosmic cop on every block scenario is
that in order for it to work it appears to be an irresistible force 
of nature, to which the ecosystem can only react. It cannot push back.

> Singletons doesn't have to be sinister Master Control Programs, they  
> could be some form of resilient oversight body implementing an  

If they cannot enforce, they're not there. If they can be attacked,
they're not there.

In order for it to work it would be the MCP from hell. For all
practical purposes, the God Hypervisor. 

> unchanging constitution. The von Neumann probe infrastructure mentioned  
> in the other thread could implement a singleton as an interface between  
> the colonizer/infrastructure construction layer and the "users",  
> essentially providing them with a DRMed galactic infrastructure. How  

You know that DRM can't work, and if you're directly accessing the physical
layer there's no virtualization. I don't see how anyone would deliberately
let themselves become somebody's serf. So you have actually have to dispatch
an overwhelming force putting everybody on leg irons.

I would know what I would do if someone would attempt that.

> perfect they need to be depends on how dangerous failures would be; the  
> more scary and brittle the situation, the more they would need to  
> prevent certain things from ever happening, but it could just be that  
> they act to bias the evolution of a civilization away from certain bad  
> attractor states like burning cosmic commons.

But there do not appear to be any cosmic commons. And before we
engage into militant astroecology, we better fix our act at home
first. If you live on this planet, you cannot reverse the Holocene
extinction. You cannot reduce your footprint to zero. You cannot
reduce your population growth.

>> I'm sure such a thing would be a dictator's wet dream.
>
> Yup. A bad singleton is an xrisk on its own.

I was wrong. There *is* a way to Blight the universe.
It's by letting loose the cosmic cops before anyone
ever had time to graduate kindergarten.



More information about the extropy-chat mailing list