[ExI] Why so much published 'science' is wrong.

[BillK]

Science is heroic, with a tragic (statistical) flaw
Mindless use of statistical testing erodes confidence in research

by Tom Siegfried  July 2, 2015 and July 10, 2015

<https://www.sciencenews.org/blog/context/science-heroic-tragic-statistical-flaw>
<https://www.sciencenews.org/blog/context/top-10-ways-save-science-its-statistical-self>

Quotes:
Too many individual papers in peer-reviewed journals are no more
reliable than public opinion polls before British elections.

More emphatically, an analysis of 100 results published in psychology
journals shows that most of them evaporated when the same study was
conducted again, as a news report in the journal Nature recently
recounted.


Numerous experts have identified statistical testing of null
hypotheses — the staple of scientific methodology — as a prime culprit
in rendering many research findings irreproducible and, perhaps more
often than not, erroneous. Many factors contribute to this abysmal
situation. In the life sciences, for instance, problems with
biological agents and reference materials are a major source of
irreproducible results, a new report in PLOS Biology shows. But
troubles with “data analysis and reporting” are also cited. As
statistician Victoria Stodden recently documented, a variety of
statistical issues lead to irreproducibility. And many of those issues
center on null hypothesis testing. Rather than furthering scientific
knowledge, null hypothesis testing virtually guarantees frequent
faulty conclusions.

A null hypothesis assumes that a factor being tested produces no
effect (or an effect no different from some other factor). If
experimental data are sufficiently unlikely (given the no-effect
assumption), scientists reject the null hypothesis and infer that
there is an effect. They call such a result “statistically
significant.”

Statistical significance has nothing to do with actual significance,
though. A statistically significant effect can be trivially small. Or
even completely illusory.

----------


I've got a feeling that this is especially relevant to ESP research,
where much of the claimed effect might probably be just statistical
creations.


BillK