[extropy-chat] Peak Oil meta-news

[Eliezer S. Yudkowsky]

Hal Finney wrote:
> 
> That's a good question.  Should the conventional wisdom, the majority
> belief, be held to the same standards as a minority or fringe viewpoint?
> 
> Look at the scientific establishment.  Most observers find a large
> degree of conservatism and hostility to new ideas, which are forced
> to meet a much higher standard of evidence than results that conform
> to the accepted paradigm.  Robin's original Idea Futures paper was
> "Could Gambling Save Science?", <http://hanson.gmu.edu/gamble.html>,
> and proposed to overcome this kind of problem by using betting markets.
> 
> Maybe science could be improved by considering all evidence equally and
> not favoring consensus views.  But maybe not!  Science has been very
> successful in terms of its goals of advancing knowledge.  It's not
> obvious a priori that its degree of conservatism is truly excessive.
> Maybe science is right to be skeptical of evidence that contravenes the
> conventional wisdom, even when the evidence looks good on its own.

Hal Finney, you're literally the only person on Earth I know who really 
lives up to Robin Hanson's ideal of modesty.  This includes Robin 
Hanson, who espouses many theories that depart from the current 
scientific consensus - not that this is a criticism of Hanson; they're 
good, interesting theories.

Doesn't it make you nervous to be the only modest person on Earth? 
Maybe modesty is wrong.  No one else believes in it.  You're not the 
least bit meta-modest in deciding how modest you ought to be; you don't 
seem to pay attention to what most scientists say about modesty, 
choosing instead to listen to Robin Hanson's fringe theories of modesty. 
  Oh, sure, Hanson's theories are much better developed mathematically 
than a few vague things Einstein and Feynman once said, but it's still 
Not The Consensus.

> The point is that in many of these disputed and fringe theories, it is
> easy to come up with a very convincing-looking case for why they are true.
> But that doesn't mean they really are true.  If you investigated every
> theory for which there exists a credible-looking story, it would take
> up all your time.
> 
> So we have a maxim: extraordinary claims require extraordinary evidence.
> Even then, it's sometimes hard to know what is an extraordinary claim.
> Robin says that medicine provides almost no quantitative benefit,
> and that sounds extraordinary, compared to what most people believe.
> Yet apparently it is the conventional wisdom in the relevant academic
> communities.
> 
> My conclusion is that it does make sense to treat fringe views
> differently from widely accepted ones.  The strategy I am following is
> to find institutions for getting at the truth that seem like they work.
> Markets are one, and science is another.  If and when those ever disagree,
> as Robin's paper anticipated, then I don't know what to do!  There are
> a few cases like that on FX, and I'm not sure what to make of them.

But is your modesty going to work *in practice*?

I had a very nice theory about a technique of human rationality - 
exemplified in the "blue tentacle" section of _Technical Explanation_ - 
which was that I should evaluate claims by asking how "unusual" they 
were, and believe the least unusual one.  I tried to use the technique 
twice to explain a surprising observation with multiple possible 
explanations, and on both occasions the technique failed me.  So I 
dropped it.  It was a nice idea but it just didn't work in practice - my 
evaluation of prior probabilities wasn't good enough.  I now reserve the 
term "extraordinary" for actual violations of physical law, or cases 
where I can actually calculate the extreme improbability.

The moral is that rationalist techniques have to work in practice. 
Rationality is not what sounds like a good idea, or what sounds like 
common sense, or even what conforms to theorems about what perfect 
rationalists would do - it is what works to arrive at the truth.

If you'd won a Nobel Prize, not by being humble before Nature, but by 
practicing your particular advocated form of modesty before scientists - 
your particular balance of credence in established institutions and 
skepticism of fringe claims - then I would say, "Behold, modesty has 
triumphed at least once."  I myself espouse a particular balance of 
credence in established institutions and skepticism of fringe claims, 
which is certainly more credulous and more skeptical than the population 
average, but still permits me to defy the mainstream once I have studied 
a matter sufficiently.  Should I enjoy significant scientific success 
(never mind doing what I actually set out to do with my life), then 
these pragmatic techniques of rationality will gain in credibility. 
It's hard for me to see how I could be beaten to the punch, outwitted 
and outgunned, by a Finneyan modesty-user of equal native intelligence.

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence