[ExI] Is psi statistics methodology wrong?

[Damien Broderick]

On 11/19/2010 4:06 PM, BillK wrote:

> One paper complains about the statistics being produced.
> Not just for the BEM results, but for all the psi testing.

 > <http://www.ruudwetzels.com/articles/Wagenmakersetal_subm.pdf>

Well, for just about *all* experimental results in *all* disciplines 
using frequentist stats. It's interesting that Bayesians regularly (one 
is tempted to say frequently) offer such critiques, but mainstream 
psychology and other disciplines show no eagerness to cast off their 
traditional means of analyzing significance.

The particular problem with Bayes applied to paradigm-challenging 
empirical results is that priors are set so extremely low that just 
about *no* results can ever get over the finishing line. (Look at their 
exemplary prior: 0.00000000000000000001.) This is not a criticism of 
Bayes, precisely, but it's something to bear in mind--especially if this 
critique is embraced with wise nods from many people who cling to 
frequentist analyses in their own work.

Damien Broderick