[extropy-chat] Re: Nano-assembler feasibility - scenarios

[Chris Phoenix]

Hal Finney wrote:
> Chris Phoenix writes:
> 
>> My first-draft scenario planning approach to addressing the 
>> uncertainties about molecular manufacturing: Use four scenarios, the 
>> product of two options: it's workable or it's not, we study it or we 
>> don't.
> 
> I see three problems with this analysis.  The first is the use ofbinary
> distinctions.  It's not just a question of whether nanotech works or not.
> There is also the issue of how long and how much effort it takes to
> get it working.  It's different if it takes 10 years, vs 100 years, vs
> 1000 years.

OK, there are three options.
1) The theory has no big holes.  In this case it could happen quite soon.
2) There's a big hole in the theory, but some kind of MM works anyway 
eventually, but it's delayed.  I think this is more or less equivalent to:
3) MM doesn't work at all.

So I don't think this affects my analysis much.

> Likewise there is no binary distinction between studying it or not.
> Rather it is a matter of how many resources we put into it.  We're
> studying it now, at a relatively low level.  Between that and some kind
> of crash Manhattan Project program there are a wide range of possible
> effort levels.  

Relatively low?  Almost non-existent!  And much of that is volunteer.

If MM is impossible, then it doesn't much matter how quickly we study 
it.  If MM is possible, the important question is: do we study it enough 
to understand and prepare for its development?  That's reasonably 
binary.  You might argue that a last-minute panic, or partial awareness, 
are interesting cases.  I might well agree.  Especially if the 
last-minute panic causes military action.

> Your scenarios don't provide much information about how
> much effort you think we should be expending.

They weren't supposed to.  They were supposed to argue that the proper 
course of action is not wait-and-see.

I personally think we should be expending at least millions per year 
specifically on studying technical issues and policy implications, 
starting today.  And the studies should give a best-guess worst-case 
report every three months, for funding to be reevaluated.  Each three 
months that the report continues to say molecular manufacturing is 
possible, and does not come up with a comforting worst-case timeline, 
the funding should be doubled.  After six months, policy theorists 
should get involved.  After twelve months if not sooner, national 
security people should get involved.

> The second problem is that in scenario 2, where we study it but it doesn't
> work, you don't count the opportunity costs from not putting resources
> into a more productive project.

Funding is not zero-sum.  This is worth increasing the deficit for, or 
buying 1% less of a Stealth bomber.

> The third problem is that in scenarios 3 and 4, you fail to account for
> the negative consequences of developing nanotech.

In scenarios 3 and 4, I'm assuming it will be developed no matter what 
we do.  I'm arguing for studying it, not for developing it.  Studies 
will probably lead to development--but failing to study it will only 
lead to development elsewhere.

> Any form of power can be both used and abused, and
> nanotech is arguably the most powerful technology that we will have
> ever developed.  The outcome is something like positive infinity plus
> negative infinity, and that makes it very hard to predict.

I agree.  And this makes it important to study rather than going in blindly.

> Your analysis is similar to Pascal's wager, where he also used two
> binary choices: God exists or not, and I believe in him or not.  If God
> exists, believing or not makes an enormous difference, dominating the
> considerations when God doesn't exist, even if it seems unlikely that
> God exists.  In your argument, if nanotech works then developing it or
> not makes an enormous difference, dominating the considerations when
> nanotech doesn't work, even if it seems unlikely that nanotech will work.

In Pascal's wager, there is no way to estimate the likelihood that God 
exists.  That invalidates the comparison.  Here, we can at least guess 
at the likelihood.  Or we can go the other way: we can say that the 
difference between scenarios 3 and 4 means that if it's more than 1% 
likely to work, we'd better study it.  Then we can answer: Do we think 
it's less than 1% likely to work?  Anyone who thinks so is welcome to 
make a bet with me.

Chris

-- 
Chris Phoenix                                  cphoenix at CRNano.org
Director of Research
Center for Responsible Nanotechnology          http://CRNano.org