[extropy-chat] Bayes, crackpots and psi

[Robin Hanson]

On 12/20/2004, Damien Broderick quoted from Robert Matthews in New Scientist:
>... by all the normal rules for assessing scientific evidence, the case
>for ESP has been made. And yet most scientists still refuse to believe the
>findings, maintaining that ESP simply does not exist. ...
>How do you arrive at that original level of belief? ... subjectivity plays
>a big role intheir day-to-day thinking. Behind closed doors they routinely
>dismiss claims for, say, some new link between cancer and diet, simply
>because they find it implausible. ... There are always auxiliary hypotheses,
>and deciding whether the evidence backs them or the theory being tested is
>just a matter of judgement. ... Subjectivity ...
>undermines the most cherished principle of the scientific process: that,
>in the end, the accumulation of evidence ensures the truth will come out.

Even given these serious problems, it does seem that eventually the truth will
come out.  But that is not a vindication of existing academic institutions -
the question is whether the truth would come out faster some other way.

Complaints about subjectivity in evaluating the support that existing evidence
gives hypotheses seem misdirected to me.  Far too much goes on inside our 
minds,
relative to our ability to communicate between minds, for us to make explicit
all our reasons for all of our beliefs.  So of course in the end a lot must
come down to non-explicit analyses and judgements.  There is no other option.

The problem in my opinion is not that subjective judgements are required, but
rather the skewed incentives to support standard vs non-standard judgements.
I am now old enough to have seen this scenario played out several times:
academic insiders have some conventional wisdom, and some group of relative
outsiders challenge that conventional wisdom.  Sometimes the outsiders are
eventually vindicated, and other times they are not.  The problem is that even
when the outsiders are vindicated, it is the insiders that mostly win.

At first the outsiders write, but cannot publish in as prestigious venues, or
get as much support.  Then at some point some insiders see that the tide will
turn, and jump on the new area.  (Insiders here are those that went to and
teach at the top schools, that know editors at top journals, etc.)  Those
insiders that first jump are able to publish in the prestigious venues, and
get substantial support, in terms or money and smart grad students to do the
grunt work.  They know the exact style that insiders require, and are invited
to write the review articles and give the talks and the media 
interviews.  Their
papers will be cited as seminal and they will get the big prizes later.  Some
acknowledgement may be made of the outsiders earlier contributions, but they
will be thought too sloppy, shrill, incomplete, informal, etc. to be 
considered
having made the crucial innovation.

Outsiders are often grateful at first, that these insiders have the 
connections,
and lingo, and style, to get their ideas taken seriously.  Some outsiders
do not realize that they will soon be forgotten - while others are willing to
accept this as the price for their ideas to win.

Now one very nice feature of this whole system is that good ideas can win -
insiders do not very strongly resist better ideas, an in fact can have
strong incentives to look around for outside ideas that deserve to win.
The main problem is that the insiders have too little incentive to create such
ideas and develop them to the point where they can become a new fashion.  This
task falls to a legion of outsiders ever optimistic about their future fame.

In some sense this story is parallel to how innovation in business works.  Many
fools think that it is enough to just have a good idea, and they send piles of
business ideas to venture capitalists.  But these VCs are mainly looking for
who would be good people to implement some idea, and if it is not the folks
who sent in the idea the VCs will happily steal the idea and give it to folks
they think could better implement it.  So most of the monetary rewards go to
those who develop the skills needed to implement good ideas, while those who
create good ideas get very little on average for their troubles.

My proposed solution is to create betting markets on scientific ideas and on
business ideas, so that those with better insight into what ideas will win can
at least be rewarded by winning their bets.



Robin Hanson  rhanson at gmu.edu  http://hanson.gmu.edu
Assistant Professor of Economics, George Mason University
MSN 1D3, Carow Hall, Fairfax VA 22030-4444
703-993-2326  FAX: 703-993-2323