[ExI] The Earth Is Round (p < .05)

Jef Allbright jef at jefallbright.net
Mon Apr 28 15:28:20 UTC 2008


Today I came across this paper which, while not new, serves to
illuminate a particularly deep, widespread and **ongoing** weakness in
the practice of rationality by those whom we might expect to know
better, especially in psychology and the social sciences.

[Copying Robin and Eliezer since I'd love to see this expanded upon in
their Overcoming Bias blog.]

Abstract:
"After 4 decades of severe criticism, the ritual of null hypothesis
significance testing—mechanical dichotomous decisions around a sacred
.05 criterion—still persists. This article reviews the problems with
this practice, including its near-universal misinterpretation of p as
the probability that H_0 is false, the misinterpretation that its
complement is the probability of successful replication, and the
mistaken assumption that if one rejects H_0 one thereby affirms the
theory that led to the test. Exploratory data analysis and the use of
graphic methods, a steady improvement in and a movement toward
standardization in measurement, an emphasis on estimating effect sizes
using confidence intervals, and the informed use of available
statistical methods is suggested. For generalization, psychologists
must finally rely, as has been done in all the older sciences, on
replication."

<http://web.math.umt.edu/wilson/Math444/Handouts/Cohen94_earth%20is%20round.pdf>

- Jef



More information about the extropy-chat mailing list