[extropy-chat] The Proactionary Principle: comments encouraged on almost-final version

[Hal Finney]

This is kind of off-topic and I don't mean for it to derail the progress
towards completion of this statement.  Certainly the principles advocated
here will be a much better foundation for future progress than anything
like the precautionary principle.

Nevertheless I couldn't help recalling our discussion last month
initiated by Robin Hanson, on the utility of scenario-based forecasting.
(Thread title was "Inside Vs. Outside Forecasts".)  Some of the advice
in the proposed document amounts to creating inside-type forecasts,
i.e. setting up scenarios, looking at probable outcomes, and making
decisions on that basis.  The paper we discussed last month shows that
this forecasting methodology is not very good, unfortunately.  It is
prone to cognitive biases of many kinds.

Unfortunately it is not clear whether there is a better alternative
for predicting the future.  As the discussion between Eliezer and
Robin elucidated, there are certain reasons to expect the future to
be like the past, and other reasons to expect it to be entirely novel
and unpredictable.  When we try to predict the effects of significant
technological innovation, should we look to the past history of "similarly
disruptive" technologies?  Or is each new situation too novel for the
past to be a useful guide?  I didn't see a consensus appear.

We didn't talk about it much, but actually the article we were discussing
was mostly on another topic that is also relevant here, the management
of risk in the corporate environment.  At a certain level, business
is all about taking risks.  Successful risk management makes or breaks
a business.  Nevertheless the article identified a number of errors in
how business executives evaluate and manage risks.  In particular, most
executives saw a major failure as being a career-ending proposition,
while an equally spectacular success would have a more modest benefit.
This causes them to be risk-averse, to the detriment of the overall
enterprise.  Another factor appears when decisions are delegated to
subordinates.  From the superior's position, failures by some of his
employees can be compensated by successes by others.  He would therefore
like to see a greater level of risk-taking.  But from the subordinate
position, failure has an overwhelming personal impact, hence he will
not take risks which are rational for the company as a whole.

(Ironically, one of the points of the article was that these risk
management errors, which cause businesses to be excessively risk-averse,
may be cancelled by inside-style scenario forecasting, which tends to
underestimate risk. This leads to an overall risk environment that may
be roughly risk-neutral, which is the economically optimal position!)

With regard to the Proactionary Principle, we are mostly looking not
at corporate risk-taking, but at government and regulatory risk issues.
This article did not go into that specifically, but there has been much
work in this area as well.  If anything, the risk environment is even
worse in the regulatory field.  A businessman who takes a successful
risk balances a significant reward against the possibility of failure,
but a regulator gets little or no reward and faces only the penalties
from failure.

The problem, then, is that even if Proactionary principles achieve
success at the policy level, the mechanisms we have for risk analysis
at the regulatory level may still undercut these policy goals by
overestimating risks.  Risk-averse regulation could result from even an
ostensibly risk-neutral policy.

On the other hand, if we could create mechanisms for a risk-neutral
regulatory environment, my feeling is that this would go a long way
towards achieving the goals of the Proactionary principle, even without
explicit policy change.  Risk-neutral regulation would by definition
take into consideration the pros and cons of any given technology or
other innovation.  That is what most of the advice in the Proactionary
Principle amounts to.

Imagine, as a toy example, if regulators were allowed to profit from
benefits created by new technologies they allow, while equally being
penalized by harm those technologies might create.  This would not be
totally risk-neutral but it would be closer than what we have today.
In such an environment I think we would immediately see innovative
technologies being treated more favorably.  In fact, I imagine that this
is why any such proposal would be opposed at higher levels, because
policy-makers understand that this would undercut their control over
the regulatory environment.

Overall, then, I agree that the Proactionary Principle is good policy.
However, without addressing the details of how society evaluates and
manages the risks of new technologies, any such policy faces obstacles to
effective implementation.  I would like to see support for such classic
Extropian ideas as Idea Futures, Delphi forecasting, science courts,
and other innovative mechanisms for more accurate forecast methodologies.
This should be part of a general research effort towards evaluating how
well different forecast mechanisms work, including traditional scenario
based forecasts.  I also think it will be necessary to look in some detail
at how policy recommendations are implemented at the regulatory level.
The regulatory risk-reward balance needs to be substantially redesigned
as part of any change towards a more risk-neutral policy stance.

Hal Finney