[extropy-chat] Superrationality (was: singularity conference at stanford)

[Hal Finney]

I commented skeptically on Doug Hofstadter's belief in superrationality,
and Jef Allbright replied:

> When I first learned about Prisoners' Dilemma -- and it was from that
> same Scientific American article -- it illustrated clearly for me that
> there was something more to real-world rationality than what was being
> dealt with in standard game theory.  This sensitivity to more
> encompassing context which is always a factor in the real world needed
> accounting for, and Hofstadter's superrationality, along with
> Buckminster Fullers statements about synergy, and other thinking on
> positive sum interactions seemed (to me) to make sense of this
> important question.
>
> I am extremely interested in knowing why you see Hofstadter's
> superrationality as wrong.

Let me first try to make the case for superrationality, and then to say
why I don't find it convincing.  Unfortunately I can't find my copy of
Hofstadter's book right now so I have to work from memory.  The argument
in favor goes like this:

The traditional Prisoner's Dilemma game is symmetric, with both parties
facing exactly the same situation.  Therefore, if we assume they are both
rational, they will come up with the same strategy for playing the game.
This means that the only possible rational outcomes are for both to
Cooperate or both to Defect.  Since the payoff for Cooperate is higher,
that is what they both should choose.

So, what's wrong with this reasoning?

Well, first of all, people don't find it convincing.  Much of Hofstadter's
column was about his frustration in trying to sell this reasoning to his
friends and colleagues.  One of the most common responses was for the
person to agree with every step up until the last.  Then they'd say,
"That means the other guy will choose to Cooperate, so obviously I
should Defect and really do well!"  Then Hofstadter would put his head
in his hands.  It was sad, but kind of funny in a way.

Realistically, if nobody finds your reasoning convincing, you have to
consider the possibility - even the likelihood - that you're wrong.
Now, granted, you can come up with many examples where people rejected
ideas which turned out to be correct, but those are rare compared to
the many bad ideas which have been rejected.  So that's the first strike
against superrationality.

The second problem is that economists reject it.  By definition,
superrationality is irrational.  That is, rational behavior has certain
mathematical characteristics.  One is that, in a game theory matrix, any
strategy which is "dominated" is eliminated.  A strategy is dominated if
there is another strategy which does better in all circumstances.  In the
PD, Cooperate is dominated by Defect.  Hence the rational thing to do is
to eliminate Cooperate as a possibility.  Defect is the rational thing
to do, which is of course where the whole literature on the PD came from.

Superrationality has never gotten any traction in the economic community
as a principle of reasoning, as far as I know.  It would require throwing
out much of the foundation of economics and game theory.  If you can't
do something as fundamental as eliminating dominated strategies, what
do you have left?  How much confidence can you have in far more subtle
analyses such as Nash equilibria?  Economists are not willing to rewrite
the rulebook like this.

I have argued in favor of accepting the academic consensus particularly
on complex and difficult issues, and this is such a case.  Those people
have given a lot more thought to foundational issues than I have, and
so I am not going to contradict them just on the basis of my own ideas.

Third, I will attempt to offer a logical rebuttal to the reasoning in
favor of superrationality.  The main thing I will note is that the
rational prescription, to Defect, is basically consistent with the
pro-superrationality reasoning.  We concluded that both parties should
do the same thing and sure enough, both parties do the same thing.
That part of the reasoning is correct.

The problem is that we also assumed implicitly that the two rational
parties both had the choice to Cooperate or Defect.  However, this is
not true, because Cooperate is dominated for rational players.  Hence,
both parties really only faced a single option, to Defect, and therefore
both parties should in fact play that strategy.

So, to repeat, the flaw in the argument is that while it put some
constraints on what rational people could do, it overlooked other
constraints on what rational people could do.  By ignoring these other
considerations, it reached a false conclusion.  Rational players must
consider all constraints on what they do, and the superrational reasoning
only looks at a single constraint.

Now, I have offered this rebuttal last, to emphasize its relative
importance.  I am not an expert on logic, game theory, or economics.
I might be wrong.  Maybe someone can point out a flaw, or otherwise find
my reasoning unconvincing.  That's fine.

I put more weight on the first two arguments, the general failure of
Hofstadter to find support for superrationality among his friends and
colleagues, and the fact that 20 years later no experts use it to explain
and analyze game theory and economic problems.  If there were really
something to it, if the logic were sound, we'd know about it by now.

Having said that, I'll tell you a true story.  Spike described
Hofstadter's contest, which he called the Luring Lottery.  (I guess it
was supposed to "Lure" people to play a certain way.)  You'd put a number
on a postcard and send it in, and the person with the largest number
would win one million dollars divided by the sum of all the numbers.

Well, I worked out that the superrational thing to do was, if there were
N participants, to play a 1 with probability 1/N, else to play a 0 (i.e.
not enter).  I'm not 100% sure this is actually correct, now, but it is
at least a plausible superrational strategy, perhaps a Schelling point.
So I actually did that!  I had a book of random numbers (back in the
70s those were still in use, computer random number generators were
not widely available).  I estimated N at about 100,000 and made up a
5-digit number; then I opened the book at random, closed my eyes and put
my finger on the page.  If I had happened to match my 5-digit number,
I would have entered a 1.

However, as expected, it did not match.  So I did not enter.  And then,
as Spike has described, Hofstadter did get his entries with enormous
numbers on them.  Whether that was the cause or not, he described the
lottery result and then announced that he was closing down his column:

"Did I find this amusing?  Somewhat, of course.  But at the same time, I
found it disturbing and disappointing.  Not that I hadn't expected it.
Indeed, it was precisely what I expected..."

And a few paragraphs later

"And with this perhaps sobering conclusion, I would like to draw my term
as a columnist for Scientific American to a close."

I actually think it would be interesting to see if Hofstadter still
believes in superrationality.  I guess I could have asked him when I had
the luck to meet him Saturday, but I didn't think of it.  Probably it
would have been hard for me to ask without seeming rude, since I would
really be asking him whether he still holds to an idea which has been
overwhelmingly rejected by almost everyone who has heard of it for
over twenty years.

Hal