[extropy-chat] Re: Nano-assembler feasibility - scenarios

[Hal Finney]

Chris Phoenix writes:

> My first-draft scenario planning approach to addressing the 
> uncertainties about molecular manufacturing: Use four scenarios, the 
> product of two options: it's workable or it's not, we study it or we 
> don't.

I see three problems with this analysis.  The first is the use ofbinary
distinctions.  It's not just a question of whether nanotech works or not.
There is also the issue of how long and how much effort it takes to
get it working.  It's different if it takes 10 years, vs 100 years, vs
1000 years.  The U.S. moon program was a large but manageable effort.
If somehow we had committed instead to sending a man to the stars and
returning him safely to earth, that would have been foolhardy in 1961.

Compare nuclear fusion power.  It's possible in theory, it would be
tremendously valuable, but we've been working on it for 50 years now
and it's still 50 years away.  Does this count as a "workable" technology?

Likewise there is no binary distinction between studying it or not.
Rather it is a matter of how many resources we put into it.  We're
studying it now, at a relatively low level.  Between that and some kind
of crash Manhattan Project program there are a wide range of possible
effort levels.  Your scenarios don't provide much information about how
much effort you think we should be expending.

The second problem is that in scenario 2, where we study it but it doesn't
work, you don't count the opportunity costs from not putting resources
into a more productive project.  Yes, we'll get some spinoffs from even
a failed nanotech effort, but Tang did not justify the space program.
Successful technology research will also produce spinoffs, plus whatever
benefits come from achieving its goal.  Working on a failed nanotech
effort means that we won't gain the benefits from whatever project we
didn't work on instead.

The third problem is that in scenarios 3 and 4, you fail to account for
the negative consequences of developing nanotech.  Yes, nano will give
enormous benefits in terms of new materials and new capabilities.  But it
also raises tremendous risks of misuse.  I'm not talking specifically
about gray goo, but the general concept of using nano for violent and
harmful purposes.  Any form of power can be both used and abused, and
nanotech is arguably the most powerful technology that we will have
ever developed.  The outcome is something like positive infinity plus
negative infinity, and that makes it very hard to predict.

Your analysis is similar to Pascal's wager, where he also used two
binary choices: God exists or not, and I believe in him or not.  If God
exists, believing or not makes an enormous difference, dominating the
considerations when God doesn't exist, even if it seems unlikely that
God exists.  In your argument, if nanotech works then developing it or
not makes an enormous difference, dominating the considerations when
nanotech doesn't work, even if it seems unlikely that nanotech will work.

Hal