[extropy-chat] three coins in a fountain, part 3: the test

[Hal Finney]

I looked a bit at the mathematics of a slightly different form of the
puzzle and got a (possibly) surprising result.

Suppose as Spike suggests we have a multiple choice test where rather than
choosing one answer we can put a "weight" number by each answer, such that
the weights for the various possible answers to a question must add to 1.
Then suppose that the score is found by multiplying together the weights
for all of the right answers.

For example, suppose you came to a question with two answers and you were
80% sure that A was right, with B having a 20% chance of being right.
You could take a chance and put all your weight on A and none on B, for
an answer of (1.0, 0.0).  Or you could perhaps split your weighting by
the probability: (0.8, 0.2).  Or you could go for an even split so that
your score would be independent of the true answer: (0.5, 0.5).

Spike's problem was complicated by the desire to beat other students,
but let's simplify if by asking how to maximize your expected score.

I didn't do a full analysis, but I tried various special cases and
the result was the same.  The best strategy is always to put 100% of
your weight on the answer that you think has the highest probability.
Splitting your weight factors can only lower your expected score.

The one exception is if there are two or more of the best answers that
have equal probability.  In that case it doesn't matter how you split
up your weights among those answers, in terms of expected score.

For example, consider a 10 question test each of which has two answers.
And suppose for each question you can make a good guess and are 80%
sure that one of the possible answers is right, with the other having
a 20% chance.  If you do (1.0, 0.0) for each question, giving 100% of
your weight to the better answer, your expected score is .8^10 or 0.107.
Suppose instead you did a (0.8, 0.2) split.  Then on average you'd get
8 of the 10 right and 2 wrong, so your score would be .8^8 * .2^2 or
0.0067, reducing your score by a factor of 16!  So you pay a very high
penalty to split up your scores even to a modest degree.

The advantage of splitting is that you reduce the variance of the
result.  By guessing 100% weight on each question, you have a 10.7%
chance of getting a score of 1 and an 89.3% chance of getting a score
of 0.  Splitting the weights 80/20 you are guaranteed not to get a 0,
and you'll probably get within a couple of orders of magnitude of your
.0067 expectataion.

Hal