[ExI] Psi (no need to read this post you already knowwhatitsays)

[Damien Broderick]

On 1/8/2010 12:07 AM, Max More wrote:

> How many negative trials were *not* reported.

By "negative trials" I assume you mean something like "runs of trials 
with outcomes that were not significantly different from mean chance 
expectation." By "not reported" I assume you mean "deceptively hidden or 
discarded." My estimate in this case: None of them. Nobody has ever 
questioned Dr. Sheldrake's probity (although some of his theories are 
pretty hard to take seriously). Well, Randi did, once, until he was 
shown to have lied.

> In Psi experiments, we
> rarely hear about "the silent evidence"

In analyses of psi experiments by anomalies researchers, actually we 
hear all the time about the likelihood and magnitude of what is termed 
"the file drawer." That's where non-significant results are supposed by 
critics to be hidden away. The reality is that the file drawer can't 
*possibly* hide sufficient dud data to account for the observations. I 
take it you have reason to doubt this; what is your evidence?

Here's Dean Radin's THE CONSCIOUS UNIVERSE (not a bad summary) on the 
file-drawer effect (selective reporting) in one major protocol, and this 
is now standard: (page 79-80)

  “Another factor that might account for the overall success of the 
ganzfeld studies was the editorial policy of professional journals, 
which tends to favor the publication of successful rather than 
unsuccessful studies.  This is the “file-drawer” effect mentioned 
earlier.  Parapsychologists were among the first to become sensitive to 
this problem, which affects all experimental domains.  In 1975 the 
Parapsychological Association’s officers adopted a policy opposing the 
selective reporting of positive outcomes.  As a result, both positive 
and negative findings have been reported at the Paraspsychological 
Association’s annual meetings and in its affiliated publications for 
over two decades.

Furthermore, a 1980 survey of parapsychologists by the skeptical British 
psychologist Susan Blackmore had confirmed that the file-drawer problem 
was not a serious issue for the ganzfeld meta-analysis.  Blackmore 
uncovered nineteen complete but unpublished ganzfeld studies.  Of those 
nineteen, seven were independently successful with odds against chance 
of twenty to one or greater.  Thus while some ganzfeld studies had not 
been published, Hyman and Honorton agreed that selective reporting was 
not an important issue in this database.

Still, because it is impossible to know how many other studies might 
have been in file drawers, it is common in meta-analyses to calculate 
how many unreported studies would be required to nullify the observed 
effects among the known studies.  For the twenty-eight direct-hit 
ganzfeld studies, this figure was 423 file-drawer experiments, a ratio 
of unreported-to-reported studies of approximately fifteen to one. 
Given the time and resources it takes to conduct a single ganzfeld 
session, let alone 423 hypotheitcal unrepoted experiments, it is not 
surprising that Hyman agreed with Honorton that the file-drawer issue 
could not plausibly account for the overall results of the psi ganzfeld 
database.  There were simply not enough experimenters around to have 
conducted those 423 studies.

Thus far, the proponent and the skeptic had agreed that the results 
could not be attributed to chance or to selective reporting practices.”>

Damien Broderick