[extropy-chat] three coins in a fountain, part 3: the test

[Eliezer S. Yudkowsky]

Hal Finney wrote:
> 
> I didn't do a full analysis, but I tried various special cases and
> the result was the same.  The best strategy is always to put 100% of
> your weight on the answer that you think has the highest probability.
> Splitting your weight factors can only lower your expected score.

Congratulations, Hal, you've just reinvented (by way of noticing its 
absence) the strictly proper scoring rule.

If you square the error of the probability assigned to the correct 
answer, as most statisticians would have you do, then your expectation 
of the resulting sum of errors has its minimum when you bet at exactly 
your anticipated probabilities.

However, as I point out in http://yudkowsky.net/bayes/technical.html - 
it turned out not to be an original observation - there are very strong 
reasons to take the logarithm of the probability assigned to the correct 
answer.  Your expectation of the sum of these logarithms has its maximum 
when you bet at exactly your anticipated probabilities, so this is also 
a strictly proper scoring rule.  The squared-error rule then becomes a 
special case which applies to Gaussian distributions.

-- 
Eliezer S. Yudkowsky                          http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence