[extropy-chat] Inside Vs. Outside Forecasts
Eliezer S. Yudkowsky
sentience at pobox.com
Wed Oct 12 00:23:33 UTC 2005
Robin Hanson wrote:
> At 02:48 PM 10/11/2005, Hal Finney wrote:
>>> The application to our forecasts about the future should be
>>> obvious: Do we overestimate technical change because we tend to
>>> take an inside view, imagining the particular process that
>>> produces some innovation, instead of an outside view, looking at
>>> how long similar innovations have
>>> taken in the past?
>> I am inclined to think that this is not the reason. I may not
>> fully understand this distinction, but it sounds like the "inside
>> view" is based on analysis from a position of some expertise and
>> detailed knowledge of the problem domain. Scenarios and
>> extrapolations are constructed based on this detailed knowledge.
> That is not how I read the distinction. It seems to me more about
> imagining a story about the process. So people basing their
> forecasts on science fiction novels would count as an inside view.
> But I won't know for sure until I can read the paper.
I've read the paper and this is correct. The inside view is based on
telling stories about the process. The outside view is based on
stupidly using the mere statistical base rate, without any attempt
whatsoever to "adjust" for the "special features" of the case.
Needless to say, the outside view works far better.
Here's one story from my notes on "Timid Choices and Bold Forecasts":
Kahneman was part of a team working out a high school
heuristics-and-biases curriculum for students in Israel. Kahneman asked
everyone present to indicate on a slip of paper how many months they
thought it would take before the project was finished - resulting range,
18 to 30 months. Kahneman then turns to an expert in curriculum
development and asks, of projects that already got this far, how long
did it take them to finish? The expert replies that only 40% ever did
finish, and of the rest, it took no less than seven years, nor more than
ten. Everyone sitting at the table knew about heuristics and biases, so
they looked at each other with pale faces - they couldn't just reject
the outside view out of hand, as most people would, though seven to ten
years was entirely unacceptable. But they didn't want to just cancel
the project. So the project struggled on and eventually concluded eight
Moral 1: The outside view always wins.
Moral 2: It requires domain expertise to take an "outside" view; you
need a pool of similar cases.
Moral 3: If no statistics are available because you're trying to
predict an event that is substantially novel relative to previous ones,
you can bloody well forget about making quantitative predictions.
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
More information about the extropy-chat