[Paleopsych] CRN: Dangers of Molecular Nanotechnology
checker at panix.com
Sat Jan 22 15:47:44 UTC 2005
Dangers of Molecular Nanotechnology
Center for Responsible Nanotechnology
[I think this is a couple of years old.]
Current Results of Our Research
These pages, marked with GREEN headings, are published for comment
and criticism. These are not our final findings; some of these
opinions will probably change.
LOG OF UPDATES
CRN Research: Overview of Current Findings
Timeline for Molecular Manufacturing
Products of Molecular Manufacturing
Benefits of Molecular Manufacturing
Dangers of Molecular Manufacturing <| YOU ARE HERE
No Simple Solutions
The Need for Early Development
The Need for International Development
Thirty Essential Nanotechnology Studies
Dangers of Molecular Manufacturing
Overview: Molecular nanotechnology (MNT) will be a significant
breakthrough, comparable perhaps to the Industrial Revolution--but
compressed into a few years. This has the potential to disrupt many
aspects of society and politics. The power of the technology may cause
two competing nations to enter a disruptive and unstable arms race.
Weapons and surveillance devices could be made small, cheap, powerful,
and very numerous. Cheap manufacturing and duplication of designs
could lead to economic upheaval. Overuse of inexpensive products could
cause widespread environmental damage. Attempts to control these
and other risks may lead to abusive restrictions, or create demand for
a black market that would be very risky and almost impossible to stop;
small nanofactories will be very easy to smuggle, and fully
dangerous. There are numerous severe risks--including several
different kinds of risk--that cannot all be prevented with the same
approach. Simple, one-track solutions cannot work. The right answer is
unlikely to evolve without careful planning.
Molecular nanotechnology suddenly will create many risks.
The potential benefits of molecular manufacturing are immense, but
so are the dangers. In order to avert the dangers, we must thoroughly
understand them, and then develop comprehensive plans to prevent them.
As explained in our Timeline and Products pages, molecular
nanotechnology (MNT) will allow the rapid prototyping and inexpensive
manufacture of a wide variety of powerful products. This capability
will arrive rather suddenly, since the final steps of developing the
technology are likely to be much easier than the initial steps, and
many of them can be pre-planned. The sudden arrival of molecular
manufacturing may not allow time to adjust to its implications.
Adequate preparation is essential.
CRN has identified several separate and severe risks.
The first step in understanding the dangers is to identify them. CRN
has begun that process here, listing and describing several separate
and severe risks. Although probably incomplete, the list is already
Economic disruption from an abundance of cheap products
Economic oppression from artificially inflated prices
Personal risk from criminal or terrorist use
Personal or social risk from abusive restrictions
Social disruption from new products/lifestyles
Unstable arms race
Collective environmental damage from unregulated products
Free-range self-replicators (gray goo) -- downgraded as a risk factor
Black market in nanotech (increases other risks)
Competing nanotech programs (increases other risks)
Attempted relinquishment (increases other risks)
Some of the dangers described here are existential risks, that
is, they may threaten the continued existence of humankind. Others
could produce significant disruption but not cause our extinction. A
combination of several risks could exacerbate the seriousness of each;
any solution must take into account its effect on other risks.
Some of these risks arise from too little regulation, and others
from too much regulation. Several different kinds of regulation will
be necessary in several different fields. An extreme or knee-jerk
response to any of these risks will create fertile ground for other
risks. The temptation to impose apparently obvious and simple
solutions to problems in isolation must be avoided. Other pages
address the possibilities for regulation; this one is concerned
with discussing and analyzing the dangers.
Disruption of the basis of economy is a strong possibility.
The purchaser of a manufactured product today is paying for its
design, raw materials, the labor and capital of manufacturing,
transportation, storage, and sales. Additional money--usually a fairly
low percentage--goes to the owners of all these businesses. If
nanofactories can produce a wide variety of products when and
where they are wanted, most of this effort will become unnecessary.
This raises several questions about the nature of a post-nanotech
economy. Will products become cheaper? Will capitalism disappear?
Will most people retire--or be unemployed? The flexibility of
nanofactory manufacturing, and the radical improvement of its
products, imply that non-nanotech products will not be able to compete
in many areas. If nanofactory technology is exclusively owned or
controlled, will this create the world's biggest monopoly, with
extreme potential for abusive anti-competitive practices? If it is
not controlled, will the availability of cheap copies mean that even
the designers and brand marketers don't get paid? Much further study
is required, but it seems clear that molecular manufacturing could
severely disrupt the present economic structure, greatly reducing the
value of many material and human resources, including much of our
current infrastructure. Despite utopian post-capitalist hopes, it is
unclear whether a workable replacement system could appear in time to
prevent the human consequences of massive job displacement.
Major investment firms are conscious of potential economic impact. In
the mainstream financial community, there is growing recognition that
nanotechnology represents a significant wave of innovation with the
potential to restructure the economy. Here, for example, is an excerpt
from an analysis prepared for investors by Credit Suisse First
Nanotechnology is a classic, general-purpose technology (GPT). Other
GPTs, including steam engines, electricity, and railroads, have been
the basis for major economic revolutions. GPTs typically start as
fairly crude technologies, with limited uses, but then rapidly spread
into new applications.
All prior GPTs have led directly to major upheavals in the
economy--the process of creative destruction. And nanotechnology may
be larger than any of the other GPTs that preceded it. Creative
destruction is the process by which a new technology or product
provides an entirely new and better solution, resulting in the
complete replacement of the original technology or product. Investors
should expect that creative destruction will not only continue, but
will also likely accelerate, and nanotechnology will be at the core.
What does this mean from a practical standpoint? Because of the
advent of nanotechnology, we believe new companies will displace a
high percentage of today's leading companies. The majority of the
companies in today's Dow Jones industrials Index are unlikely to be
there 20 years from now. (Excerpted with permission from "Big Money in
Thinking Small", authored by Michael Mauboussin and Kristen
Along those same lines, Josh Wolfe of Lux Capital, editor of the
Forbes/Wolfe Nanotech Report, writes: "Quite simply, the world is
about to be rebuilt (and improved) from the atom up. That means tens
of trillions of dollars to be spent on everything: clothing... food...
cars... housing... medicine...the devices we use to communicate and
recreate...the quality of the air we breathe...and the water we drink,
are all about to undergo profound and fundamental change. And as a
result, so will the socio and economic structure of the world.
Nanotechnology will shake up just about every business on the planet."
MNT products may be vastly overpriced relative to their cost,
perpetuating unnecessary poverty.
By today's commercial standards, products of MNT would be
immensely valuable. A monopoly would allow the owners of the
technology to charge high rates for all products, and make high
profits. However, if carried to its logical conclusion, such a
practice would deny cheap lifesaving technologies (as simple as water
filters or mosquito netting) to millions of people in desperate need.
Competition will eventually drive prices down, but an early monopoly
is likely for several reasons. Due to other risks listed on this page,
it is unlikely that a completely unregulated commercial market will be
allowed to exist. In any case, the high cost of development will limit
the number of competing projects. Finally, a company that pulls ahead
of the pack could use the resulting huge profits to stifle competition
by means such as broad enforcement of expansive patents and lobbying
for special-interest industry restrictions.
The price of a product usually falls somewhere between its value to
the purchaser and its cost to the seller. Molecular manufacturing
could result in products with a value orders of magnitude higher than
their cost. It is likely that the price will be set closer to the
value than to the cost; in this case, customers will be unable to gain
most of the benefit of "the nanotech revolution". If pricing products
by their value is accepted, the poorest people may continue to die of
poverty, in a world where products costing literally a few cents would
save a life. If (as seems likely) this situation is accepted more by
the rich than by the poor, social unrest could add its problems to
untold unnecessary human suffering. A present-day example is the
agreement that the WTO was working on to provide affordable medicines
to poor countries--which the Bush administration partially
prevented (following heavy lobbying by American pharmaceutical
companies) despite furious opposition from every other WTO member.
Criminals and terrorists could make effective use of the technology.
Criminals and terrorists with stronger, more powerful, and much more
compact devices could do serious damage to society. Defenses against
these devices may not be installed immediately or comprehensively.
Terrorists could have a field day. Chemical and biological weapons
could become much more deadly and much easier to conceal. Many other
types of terrifying devices are possible, including several varieties
of remote assassination weapons that would be difficult to detect or
avoid. If such devices were available from a black market or a home
factory, it would be quite difficult to detect them before they were
used; a random search capable of spotting them would be a clear
violation of current human rights standards in most civilized
countries. Detecting a criminal user after the fact might also be
difficult; since many devices can be computer-controlled and
networked, the criminal does not have to be at the scene.
Extreme solutions and abusive regulations may be attempted.
A patchwork of extreme solutions may be created in response to the
other risks described here. This would not be a good idea. Many of
these problems appear to have an obvious solution. However, in each
case, that solution, applied to the extreme necessary to impact the
target problem, would exacerbate another problem and make the overall
situation worse. A collection of extreme solutions will surely be
undesirable; it will either be ineffective (and ineffective policies
can still be quite harmful) or will create massive human suffering or
human rights violation.
There is a possibility that abusive restrictions and policies may be
attempted, such as round-the-clock surveillance of every citizen.
Such surveillance might be possible with AI programs similar to one
currently being developed at MIT, which is able to analyze a video
feed, learn familiar patterns, and notice unfamiliar patterns.
Molecular manufacturing will allow the creation of very small,
inexpensive supercomputers that conceivably could run a program of
constant surveillance on everyone. Surveillance devices would be easy
to manufacture cheaply in quantity. Surveillance is only one possible
kind of abuse. With the ability to build billions of devices, each
with millions of parts, for a total cost of a few dollars, any
automated technology that can be applied to one person can be applied
to everyone. Any scenario of physical or psychiatric control that
explores the limits of nanotechnology will sound science-fictional and
implausible. The point is not the plausibility of any given scenario;
it is that the range of possibilities is limited mainly by the
imagination and cruelty of those with power. Greed and power are
strong motivators for abusive levels of control; the fear of nanotech
and other advanced technologies in private hands adds an additional
impetus for abusive rule.
Society could be disrupted by the availability of new "immoral"
New products and lifestyles may cause significant social disruption.
For example, medical devices could be built into needles narrower than
a bacterium, perhaps allowing easy brain modification or stimulation,
with effects similar to any of a variety of psychoactives. Most
societies have found it desirable to forbid certain products: guns in
Britain, seedless watermelon in Iran, sex toys in Texas, various drugs
in various societies such as hashish in the United States and alcohol
in Muslim societies. Although many of these restrictions are based on
moral principles not shared by the majority of the world's population,
the fact that the restrictions exist at all indicates the sensitivity
of societies--or at least their rulers--to undesired products. The
ability to make banned products using personal factories could be
expected to be at least somewhat disruptive to society, and could
provide an impetus for knee-jerk and overly broad restrictions on the
technology. New lifestyles enabled by new technology could also cause
social disruption. Whereas demand for banned products already exists,
lifestyles develop over time, so the effects of lifestyle change are
likely to be less acute. However, some lifestyle possibilities
(particularly in the areas of sex, drugs, entertainment, and body or
genetic modification) are likely to be sufficiently disturbing to
onlookers that their very existence would cause disruption.
Nanotech weapons would be extremely powerful and could lead to a
dangerously unstable arms race.
Molecular manufacturing raises the possibility of horrifically
effective weapons. As an example, the smallest insect is about 200
microns; this creates a plausible size estimate for a
nanotech-built antipersonnel weapon capable of seeking and injecting
toxin into unprotected humans. The human lethal dose of botulism toxin
is about 100 nanograms, or about 1/100 the volume of the weapon. As
many as 50 billion toxin-carrying devices--theoretically enough to
kill every human on earth--could be packed into a single suitcase.
Guns of all sizes would be far more powerful, and their bullets could
be self-guided. Aerospace hardware would be far lighter and higher
performance; built with minimal or no metal, it would be much harder
to spot on radar. Embedded computers would allow remote activation of
any weapon, and more compact power handling would allow greatly
improved robotics. These ideas barely scratch the surface of what's
An important question is whether nanotech weapons would be
stabilizing or destabilizing. Nuclear weapons, for example, perhaps
can be credited with preventing major wars since their invention.
However, nanotech weapons are not very similar to nuclear weapons.
Nuclear stability stems from at least four factors. The most obvious
is the massive destructiveness of all-out nuclear war. All-out
nanotech war is probably equivalent in the short term, but nuclear
weapons also have a high long-term cost of use (fallout,
contamination) that would be much lower with nanotech weapons. Nuclear
weapons cause indiscriminate destruction; nanotech weapons could be
targeted. Nuclear weapons require massive research effort and
industrial development, which can be tracked far more easily than
nanotech weapons development; nanotech weapons can be developed much
more rapidly due to faster, cheaper prototyping. Finally, nuclear
weapons cannot easily be delivered in advance of being used; the
opposite is true of nanotech. Greater uncertainty of the capabilities
of the adversary, less response time to an attack, and better targeted
destruction of the enemy's resources during an attack all make
nanotech arms races less stable. Also, unless nanotech is tightly
controlled, the number of nanotech nations in the world could be much
higher than the number of nuclear nations, increasing the chance of a
regional conflict blowing up.
Admiral David E. Jeremiah, Vice-Chairman (ret.), U.S. Joint
Chiefs of Staff, in an address at the 1995 Foresight Conference on
Molecular Nanotechnology said: "Military applications of molecular
manufacturing have even greater potential than nuclear weapons to
radically change the balance of power."
An excellent essay by Tom McCarthy (unaffiliated with CRN)
explores these points in more detail. He discusses the ways that MNT
can destabilize international relations: MNT will reduce economic
influence and interdependence, encourage targeting of people as
opposed to factories and weapons, and reduce the ability of a nation
to monitor its potential enemies. It may also, by enabling many
nations to be globally destructive, eliminate the ability of powerful
nations to "police" the international arena. By making small groups
self-sufficient, it can encourage the breakup of existing nations.
Collective environmental damage is a natural consequence of cheap
(MORE) Molecular manufacturing allows the cheap creation of
incredibly powerful devices and products. How many of these products
will we want? What environmental damage will they do? The range of
possible damage is vast, from personal low-flying supersonic aircraft
injuring large numbers of animals to collection of solar energy on a
sufficiently large scale to modify the planet's albedo and directly
affect the environment. Stronger materials will allow the creation of
much larger machines, capable of excavating or otherwise destroying
large areas of the planet at a greatly accelerated pace. It is too
early to tell whether there will be economic incentive to do this.
However, given the large number of activities and purposes that would
damage the environment if taken to extremes, and the ease of taking
them to extremes with molecular manufacturing, it seems likely that
this problem is worth worrying about. Some forms of damage can result
from an aggregate of individual actions, each almost harmless by
itself. Such damage is quite hard to prevent by persuasion, and laws
frequently don't work either; centralized restriction on the
technology itself may be a necessary part of the solution. Finally,
the extreme compactness of nanomanufactured machinery will tempt the
use of very small products, which can easily turn into nano-litter
that will be hard to clean up and may cause health problems.
Gray goo was an early concern of nanotechnology. When
nanotechnology-based manufacturing was first proposed, a concern arose
that tiny manufacturing systems might run amok and 'eat' the
biosphere, reducing it to copies of themselves. In 1986, Eric Drexler
wrote, "We cannot afford certain kinds of accidents with replicating
assemblers." More recent designs by Drexler and others make it clear,
though, that replicating assemblers will not be used for
manufacturing--nanofactories will be much more efficient at
building products, and a nanofactory is nothing like a 'gray goo'
Gray goo would entail five capabilities integrated into one small
package. These capabilities are: Mobility - the ability to travel
through the environment; Shell - a thin but effective barrier to keep
out diverse chemicals and ultraviolet light; Control - a complete set
of blueprints and the computers to interpret them (even working at the
nanoscale, this will take significant space); Metabolism - breaking
down random chemicals into simple feedstock; and Fabrication - turning
feedstock into nanosystems. A nanofactory would use tiny
fabricators, but these would be inert if removed or unplugged from
the factory. The rest of the listed requirements would require
substantial engineering and integration.
Gray goo won't happen by accident, but eventually could be developed
on purpose. Although gray goo has essentially no military and no
commercial value, and only limited terrorist value, it could be used
as a tool for blackmail. Cleaning up a single gray goo outbreak would
be quite expensive and might require severe physical disruption of the
area of the outbreak (atmospheric and oceanic goos deserve special
concern for this reason). Another possible source of gray goo release
is irresponsible hobbyists. The challenge of creating and releasing a
self-replicating entity apparently is irresistible to a certain
personality type, as shown by the large number of computer viruses and
worms in existence. We probably cannot tolerate a community of
"script kiddies" releasing many modified versions of goo.
Development and use of molecular manufacturing poses absolutely no
risk of creating gray goo by accident at any point. However, goo type
systems do not appear to be ruled out by the laws of physics, and we
cannot ignore the possibility that the five stated requirements could
be combined deliberately at some point, in a device small enough that
cleanup would be costly and difficult. Drexler's 1986 statement can
therefore be updated: We cannot afford criminally irresponsible misuse
of powerful technologies. Having lived with the threat of nuclear
weapons for half a century, we already know that.
We wish we could take gray goo off CRN's list of dangers, but we
can't. It eventually may become a concern requiring special policy.
Gray goo will be highly difficult to build, however, and
non-replicating nano-weaponry may be substantially more dangerous
and more imminent. Since there are greater risks from molecular
manufacturing (such as falling into an unstable arms race) that may
happen almost immediately after the technology is developed, CRN does
not see gray goo as a primary concern.
UPDATE: In June 2004, Eric Drexler and Chris Phoenix published
a new paper on "Safe Exponential Manufacturing", which puts the
perceived gray goo threat into perspective.
Too little or too much regulation can result in unrestricted
Uncontrolled availability of MNT can result from either insufficient
or overzealous regulation. Inadequate regulation would make it easy to
obtain and use an unrestricted nanofactory. Overzealous regulation
would create a pent-up demand for nanotech products, which if it gets
strong enough, would fund espionage, cracking of restricted
technology, or independent development, and eventually create a black
market beyond the control of central authorities (nanofactories are
very smugglable). Note that sufficiently abusive or restrictive
regulation can motivate internal espionage; at least one atomic spy in
the US was idealistically motivated. Uncontrolled availability of
molecular manufacturing greatly increases many of the dangers cited
Competing nanotech programs increase the danger.
The existence of multiple MNT programs greatly increases some of the
risks listed above. Each program provides a separate opportunity for
the technology to be stolen or otherwise released from restriction.
Each nation with an independent program is potentially a separate
player in a nanotech arms race. The reduced opportunity for control
may make restrictions harder to enforce, but this may lead to greater
efforts to impose harsher restrictions. Reduced control also makes it
less likely that a non-disruptive economic solution can develop.
Relinquishment is counterproductive.
Facing all these risks, there will be a strong temptation simply to
outlaw the technology. However, we don't believe this can work. Many
nations are already spending millions on basic nanotechnology; within
a decade, advanced nanotech will likely be within the reach of large
corporations. It can't be outlawed worldwide. And if the most
risk-aware countries stop working on it, then the less responsible
countries are the ones that will be developing it and dealing with it.
Besides, legal regulation may not have much effect on covert military
MNT may be delayed by strict regulation, but this would probably
make things worse in the long run. If MNT development is delayed until
it's relatively easy, it will then be a lot harder to keep track of
all the development programs. Also, with a more advanced technology
base, the development of MNT products could happen even faster than we
have described, leaving less time to adjust to the societal
Solving these problems won't be easy.
Some of these risks arise from too little regulation, and others from
too much regulation. Several different kinds of regulation will be
necessary in several different fields. An extreme or knee-jerk
response to any of these risks will simply create fertile ground for
other risks. The risks are of several different types, so a single
approach (commercial, military, free-information) cannot prevent all
of them. Some of the risks are sufficiently extreme that society
cannot adjust to the risk while testing various approaches to prevent
it. A single gray goo release, or unstable nanotech arms race, is
intolerable. Threading a path between all these risks will require
careful advance planning
DEVIL'S ADVOCATE -- Submit your criticism, please!
You're assuming only the bad guys will have nanotech.
No. We're assuming that some bad, or just irresponsible, groups
might get nanotech and misuse it before every good guy has all the
technology they need to prevent every problem. In some cases the
counter-technology won't be invented yet. And even if the
counter-technology does exist, it probably won't be used as widely
as it should be--like computer anti-virus programs today.
But this is decades in the future.
We think it's less than twenty years off--maybe less than fifteen.
That's not much time to analyze the problems, and then design and
implement solutions. If you knew that a year from now, you would
have to walk a tightrope without a net, how soon would you start
Nanotech won't really be that sudden or dangerous.
Read our Timeline and Products pages to see why we think it
(From Michael Vassar) Japan managed to prohibit guns for centuries,
before Perry's gunboat finally forced a change.
Primitive guns did not confer an overwhelming military advantage.
Each gun required highly skilled labor and much time to make. No
one could hope to take over the country even with guns, and they
would have been destroyed by the Emperor for trying. By contrast,
once the first nanofactory is made, it will be pretty easy to use
and very easy to duplicate, and will provide an immense military
advantage to its owners.
This whole analysis is really too simplistic.
Well, we have to start somewhere. Please contact us and tell us
what we're missing; we promise to listen to and think about all
68. mailto:cphoenix at CRNano.org?subject=Devil's%20Advocate
More information about the paleopsych