[extropy-chat] Inside Vs. Outside Forecasts

[Eliezer S. Yudkowsky]

Robin Hanson wrote:
> At 02:48 PM 10/11/2005, Hal Finney wrote:
> 
>>> The application to our forecasts about the future should be
>>> obvious: Do we overestimate technical change because we tend to
>>> take an inside view, imagining the particular process that
>>> produces some innovation, instead of an outside view, looking at
>>> how long similar innovations have
>>> taken in the past?
>> 
>> I am inclined to think that this is not the reason.  I may not
>> fully understand this distinction, but it sounds like the "inside
>> view" is based on analysis from a position of some expertise and
>> detailed knowledge of the problem domain.  Scenarios and
>> extrapolations are constructed based on this detailed knowledge.
> 
> That is not how I read the distinction.  It seems to me more about 
> imagining a story about the process.  So people basing their
> forecasts on science fiction novels would count as an inside view.
> But I won't know for sure until I can read the paper.

I've read the paper and this is correct.  The inside view is based on 
telling stories about the process.  The outside view is based on 
stupidly using the mere statistical base rate, without any attempt 
whatsoever to "adjust" for the "special features" of the case.

Needless to say, the outside view works far better.

Here's one story from my notes on "Timid Choices and Bold Forecasts": 
Kahneman was part of a team working out a high school 
heuristics-and-biases curriculum for students in Israel.  Kahneman asked 
everyone present to indicate on a slip of paper how many months they 
thought it would take before the project was finished - resulting range, 
18 to 30 months.  Kahneman then turns to an expert in curriculum 
development and asks, of projects that already got this far, how long 
did it take them to finish?  The expert replies that only 40% ever did 
finish, and of the rest, it took no less than seven years, nor more than 
ten.  Everyone sitting at the table knew about heuristics and biases, so 
they looked at each other with pale faces - they couldn't just reject 
the outside view out of hand, as most people would, though seven to ten 
years was entirely unacceptable.  But they didn't want to just cancel 
the project.  So the project struggled on and eventually concluded eight 
years later.

Moral 1:  The outside view always wins.

Moral 2:  It requires domain expertise to take an "outside" view; you 
need a pool of similar cases.

Moral 3:  If no statistics are available because you're trying to 
predict an event that is substantially novel relative to previous ones, 
you can bloody well forget about making quantitative predictions.

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence