[ExI] psi or bad statistics?
Eliezer S. Yudkowsky
sentience at pobox.com
Mon Dec 17 23:43:14 UTC 2007
Damien Broderick wrote:
> Well, here's an extract; the doubtful will need to go to the primary sources:
> <[Dr. Dean] Radin used one of the simplest possible methods,
> measuring shifts in skin conductivity in fingers or palms when the
> wired subjects were affronted by computer images of violent or erotic
> scenes, and those reactions in turn were compared with the physical
> response elicited by soothing images and no images at all. When the
> data from many subjects were added together and averaged, in order to
> remove idiosyncratic responses and the intrusion of random noise, it
> turned out that the average response (presponse) to neutral or
> pleasant images followed pretty much the curve one would expect. The
> image flashes on the screen for three seconds, and while subjects
> watch the blank screen that follows, their skin conductivity rises
> slightly, drops away again, flutters along in its normal quietly
> wandering path.>
It's a remarkable, amazing fact about psychic powers that they seem to
work as well backward in time as forward; as well for psi-miss as
psi-hit; as well for manipulation as prediction; as well when the
telekinesis is exerted after the experiment as before it; and of
course, the effect size gets smaller and smaller (but still
statistically significant) as the samples get larger and larger.
Now it seems unlikely that anything a brain can really do would have
such properties, but they are all naturally expected in association
with the amazing magical power known as "bad statistics", which works
as well backward in time as forward, as well for misses as hits, etc.
The particular statistical flaw in Radin's experiments would seem to
be described here:
It's amazing what subtle statistical flaws you can uncover when you
know a priori that the effect is not real. This problem would almost
certainly had gone undetected if Radin had been performing a more
conventional medical experiment.
The psionicists have a legitimate complaint that they are being held
to higher standards than the rest of science. Bad statistics are an
increasingly huge problem for the rest of science, too, but unlike
psionics we don't know a priori that the detected effects are unreal.
For example, it's now suspected that half of all published medical
studies in major journals have irreproduceable results.
should hold all science to the same strict standards that would be
required to eliminate ESP.
I've been advocating that p<0.001 should replace p<0.05 as the margin
of statistical significance. Physics journals routinely require
p<0.0001. It would be better by far to do fewer medical experiments
with more subjects and have nearly all the published reports be valid.
But there's a Nash equilibrium for the bad behavior, where you use
the lower standard and make sure of a publication.
I myself now put no more than a 50% probability on any published
finding that is statistically significant at p<0.05 rather than
p<0.001. Less, if the finding seems iffy in other ways.
Eliezer S. Yudkowsky http://singinst.org/
Research Fellow, Singularity Institute for Artificial Intelligence
More information about the extropy-chat