[ExI] Is psi statistics methodology wrong?

BillK pharos at gmail.com
Fri Nov 19 22:06:01 UTC 2010


Now that every man and his dog on the interweb are talking about the
latest BEM precognition results, some skeptics have started to respond
with criticisms of the experiments.

One paper complains about the statistics being produced.
Not just for the BEM results, but for all the psi testing.

<http://www.ruudwetzels.com/articles/Wagenmakersetal_subm.pdf>

It is quite a complex paper and probably needs a stats expert to fully
understand what they are getting at.
The paper is a 12 page pdf file, but here is the Abstract:

Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi
Eric-Jan Wagenmakers, Ruud Wetzels, Denny Borsboom, & Han van der Maas
University of Amsterdam

Abstract
Does psi exist? In a recent article, Dr. Bem conducted nine studies with
over a thousand participants in an attempt to demonstrate that future events
retroactively affect people's respones. Here we discuss several limitations
of Bem's experiments on psi; in particular, we show that the data analy-
sis was partly exploratory, and that one-sided p-values may overstate the
statistical evidence against the null hypothesis. We reanalyze Bem's data
using a default Bayesian t-test and show that the evidence for psi is weak
to nonexistent. We argue that in order to convince a skeptical audience of a
controversial claim, one needs to conduct strictly confirmatory studies and
analyze the results with statistical tests that are conservative rather than
liberal. We conclude that Bem's p-values do not indicate evidence in favor
of precognition; instead, they indicate that experimental psychologists need
to change the way they conduct their experiments and analyze their data.
----------------


BillK



More information about the extropy-chat mailing list