[extropy-chat] Wisdom of Crowds

[Hal Finney]

Thanks for changing the subject line, Bill, I was guilty of topic drift there.

BillK writes:

> It is correct to claim that the crowd *sometimes* can be a useful
> predictor. But to get a good result you have to be very careful about
> the selection of the crowd, the selection of the possible results, and
> control of the crowd behaviour.
>
> Michael Shermer reviewed this book in Dec 2004
> <http://www.sciam.com/article.cfm?chanID=sa006&articleID=00049F3E-91E1-119B-8EA483414B7FFE9F&colID=13>

Yes, that's a good review, although short.  Shermer basically reiterates
a few of the points Surowiecki makes.

> Everybody can think of cases that disprove the wisdom of crowds.
> Surowiecki mentions some of them in his book. Earlier books have put
> the opposite case. Mackay's 'Extraordinary Popular Delusions and the
> Madness of Crowds' include the well known tulipmania phenomenon.
> Canetti wrote 'Crowds and Power' with the shouts of Hitler's Nuremberg
> rally figuratively ringing in his ears.  Also sociologists such as
> Gustave Le Bon, in his classic work The Crowd: A Study of the Popular
> Mind: "In crowds it is stupidity and not mother wit that is
> accumulated."
> Lynch mobs are another example.

I posted a couple of months ago some thoughts about the book and how it
applies to famous cases of crowd hysteria:

http://lists.extropy.org/pipermail/extropy-chat/2006-March/025620.html

The basic problem is the one I mentioned yesterday, a sort of paradox
of rationality.  On the one hand, crowds frequently do very well,
and a rational person will believe the consensus.  But on the other
hand, if too many people follow this prescription, the crowd falls into
self-perpetuating and ungrounded beliefs.  These cases above that Bill
mentions, plus well known market bubbles and crashes, can be seen as
this kind of failure.

I heard a simple example of this recently (can't remember where, maybe
Kurzweil's book).  Imagine someone who goes to the race track eager to bet
on a particular horse he's been reading about.  But when he gets there,
he looks up at the board and sees that it is running at odds of 100 to
1 against!  That mere fact in itself, without any more information, is
likely to make him less desirous of betting on the horse.  The market
consensus influences his private opinion.

Economics teaches us that, modulo certain assumptions, this should
happen to a much greater degree, to the point where it is basically
impossible to disagree with the market consensus.  There is something of
an unrecognized public goods problem here.  Bets that disagree with the
market are harmful to the bettors but helpful to society.  Luckily, people
haven't yet figured out this effect, so markets generally work well.

The display of caution at the Singularity conference was an example
of what would happen if everyone started behaving rationally, and it
was a disaster!  (In the sense that we didn't get a meaningful picture
of the consensus judgement of the presenters on an important issue.)
If that starts to happen we'll need public service announcements saying,
Think for yourself!  Don't follow the crowd!

And actually, in a way, we do have that.  Not as PSAs as such, but it
is an important cultural lesson in the West.  Probably everyone has
heard advice such as this, growing up.  "If everyone else was jumping
off a cliff, would you?"  Well, frankly, if everyone was doing that,
maybe it's because it is the best of a bad situation, and in fact it's
the right thing to do.  But if everyone thinks that way and follows the
crowd, we see the kinds of bad phenomena that Bill describes above.

Hal