[extropy-chat] three coins in a fountain, part 3: the test
spike
spike66 at comcast.net
Sat Oct 22 23:02:27 UTC 2005
> bounces at lists.extropy.org] On Behalf Of "Hal Finney"
> Subject: Re: [extropy-chat] three coins in a fountain, part 3: the test
>
> I looked a bit at the mathematics of a slightly different form of the
> puzzle and got a (possibly) surprising result...
>
...
>
> For example, consider a 10 question test each of which has two answers.
> And suppose for each question you can make a good guess and are 80%
> sure that one of the possible answers is right, with the other having
> a 20% chance. If you do (1.0, 0.0) for each question, giving 100% of
> your weight to the better answer, your expected score is .8^10 or 0.107.
> Suppose instead you did a (0.8, 0.2) split. Then on average you'd get
> 8 of the 10 right and 2 wrong, so your score would be .8^8 * .2^2 or
> 0.0067, reducing your score by a factor of 16! So you pay a very high
> penalty to split up your scores even to a modest degree... Hal
Oh dear, Hal my good man, I do disagree waaaay much. Since
the game requires taking the product of the scores, if you
estimate each answer to 80% confidence, then put *all*
your chips on that choice, you would miss at least one of
question (1-.8^10) or ~89% of the time. You would get a fat
zippo most of the time. The remaining 11% of the games
you would get a perfect 1 score. But one can almost hear
in the background that oo-oo-oo sound that monkeys make
when they get excited, because they know that they will
beat you 8 of 9 times, which is unacceptable, wretched
random beasts that they are.
While I agree that your *average* score would go down
slightly if you assigned a .99 weight to your favorite
choice with .01 on the other, at least then you would
defeat the illiterate monkeys 37% of the time. On
those ~11% of the times when you were right on all,
your score would be about .904, still very impressive.
Regarding your expectation calc, I am not sure offhand
why it disagrees with my sim, but that sim suggests
an expectation of about .02 instead of .0067 with
the 80-20 split. So splitting 80-20 costs you a
factor of about 5 instead of 16, but at least you beat
the monkeys about 87% of the time.
I guess we must decide then exactly what we are trying
to optimize. My criterion is arbitrary I admit: we
will not accept losing to monkeys. If we look at it
as a competition with other students, we do not
wish to be tied for dead last 89% of the time, even
if we tie for first the rest of the time. I will
need some game theorists to help me here. Perhaps
if there is some payoff matrix, we could derive the
optimal strategy. I don't think it is all-or-nothing
in this case. I have seen where the payoff matrix
can affect optimal strategy in poker.
Far from being a meaningless exercise, this question
has a direct and critically important real world
application in the space business. With satellites,
we often have a situation where you have a number
of subsystems that must all work perfectly, otherwise
the mission is lost. We often make significant
engineering sacrifices in order to increase the
reliability of a subsystem, depending on the
consequences of a single point failure.
More information about the extropy-chat
mailing list