[extropy-chat] three coins in a fountain, part 3: the test

[spike]

> bounces at lists.extropy.org] On Behalf Of "Hal Finney"
> Subject: Re: [extropy-chat] three coins in a fountain, part 3: the test
> 
> I looked a bit at the mathematics of a slightly different form of the
> puzzle and got a (possibly) surprising result...
> 
...
> 
> For example, consider a 10 question test each of which has two answers.
> And suppose for each question you can make a good guess and are 80%
> sure that one of the possible answers is right, with the other having
> a 20% chance.  If you do (1.0, 0.0) for each question, giving 100% of
> your weight to the better answer, your expected score is .8^10 or 0.107.
> Suppose instead you did a (0.8, 0.2) split.  Then on average you'd get
> 8 of the 10 right and 2 wrong, so your score would be .8^8 * .2^2 or
> 0.0067, reducing your score by a factor of 16!  So you pay a very high
> penalty to split up your scores even to a modest degree... Hal


Oh dear, Hal my good man, I do disagree waaaay much.  Since
the game requires taking the product of the scores, if you 
estimate each answer to 80% confidence, then put *all*
your chips on that choice, you would miss at least one of
question (1-.8^10) or ~89% of the time.  You would get a fat
zippo most of the time.  The remaining 11% of the games
you would get a perfect 1 score.  But one can almost hear
in the background that oo-oo-oo sound that monkeys make
when they get excited, because they know that they will
beat you 8 of 9 times, which is unacceptable, wretched
random beasts that they are.

While I agree that your *average* score would go down
slightly if you assigned a .99 weight to your favorite
choice with .01 on the other, at least then you would
defeat the illiterate monkeys 37% of the time.  On
those ~11% of the times when you were right on all,
your score would be about .904, still very impressive.

Regarding your expectation calc, I am not sure offhand
why it disagrees with my sim, but that sim suggests
an expectation of about .02 instead of .0067 with
the 80-20 split.  So splitting 80-20 costs you a
factor of about 5 instead of 16, but at least you beat
the monkeys about 87% of the time. 

I guess we must decide then exactly what we are trying
to optimize.  My criterion is arbitrary I admit: we
will not accept losing to monkeys.  If we look at it
as a competition with other students, we do not
wish to be tied for dead last 89% of the time, even
if we tie for first the rest of the time.  I will
need some game theorists to help me here.  Perhaps 
if there is some payoff matrix, we could derive the
optimal strategy.  I don't think it is all-or-nothing
in this case.  I have seen where the payoff matrix
can affect optimal strategy in poker.

Far from being a meaningless exercise, this question
has a direct and critically important real world
application in the space business.  With satellites,
we often have a situation where you have a number
of subsystems that must all work perfectly, otherwise
the mission is lost.  We often make significant
engineering sacrifices in order to increase the 
reliability of a subsystem, depending on the 
consequences of a single point failure.