[extropy-chat] calling all bayesians

[spike]

> -----Original Message-----
> From: extropy-chat-bounces at lists.extropy.org [mailto:extropy-chat-
> bounces at lists.extropy.org] On Behalf Of Eliezer S. Yudkowsky
> Sent: Wednesday, May 11, 2005 8:57 PM
> To: ExI chat list
> Subject: Re: [extropy-chat] calling all bayesians
> 
> spike wrote:
> > Guys help me eff this real-life effing problem:
> >
> > I build 150 droobs and use 131 of them in my freem.  I
> > test the remaining 19 spares destructively and find that
> > all are good.  From that information only, what is the
> > probability that all 131 droobs are good?
> >
> > I have four Monte Carlo sims chewing on this problem
> > but they are giving me puzzling results.  A closed-form
> > solution to this would be impressive, winning my
> > undying respect.
> 
> Depends on your prior...

Wooo hooo!  This is what I hoped you would say.  I was puzzled
by the fact that any line of reasoning about this problem seems
to double back on itself.

Today at the office I argued that this problem has not enough
information to solve by itself.


>...In real life, Spike, your problem is pretty much undefined,
> unless you can give me some kind of base rate statistics on how often your
> manufacturing technique works...

Eeeeexcellent.  I argued today that we need the theoretical
reliability from the math model in order to calculate the
probability that the remaining 131 droobs are good.   


> 
> How on Earth did you set up a Monte Carlo sim on this?
> 
> --
> Eliezer S. Yudkowsky 

I set the MC to show that if one assumes one of every ten droobs
is bad, then one can calculate the probability of choosing
19 good droobs vs the probability of 131 good droobs.  Then
set the probability to one in eleven droobs are bad, repeat.
Keep going up thru one in 400 droobs is bad.  In each case,
you supply a prior, the MC hands you back probability that
all 131 remaining droobs are good.  You can do that in
closed form, so I wanted to demonstrate that the sim
agrees with theory.

This technique derives a probability distribution function
that can then be compared to the theoretical reliability
model, from which I can give back an answer.

Nowthen, here is the interesting part.  A freem only requires 
130 droobs, not 131.  A failed droob was discovered in the 130, 
so it was replaced by one of the 20 spares.  So that leaves 19 
spares and possibly some information regarding the reliability
of a droob.  If I assume the theoretical reliability of
the droobs at one bad in 130, then the MC sim gives an answer
for the probability that the remaining 130 are all good (~37%).
Testing 19 and finding all good tells me almost nothing, because
that is the expected outcome (~86%).  But without further info,
I don't know that droobian reliability is one in 130 bad.  I
fall into a kind of circular reasoning.

Does this conclusion agree with a Bayesian approach?