[ExI] [Bulk] Why so much published 'science' is wrong.

[spike]

>... On Behalf Of BillK
...

>...Statistical significance has nothing to do with actual significance, though. A statistically significant effect can be trivially small. Or even completely illusory.

----------
>...I've got a feeling that this is especially relevant to ESP research, where much of the claimed effect might probably be just statistical creations.  BillK
_______________________________________________


BillK, in the engineering world, there is an appalling lack of understanding on what the threshold of statistical significance means and what it doesn't mean.  Looking back on my own formal education in the topic, they didn't really explain sufficiently the meaning of statistical significance.  There is far too much mindless plugging into a formula without sufficient understanding of what the formula is saying (if anything.)

If someone wants a good scientific study, here it is: create a number of datasets on some known (to the experimenter) function.  Then go to some highly respected science-geek university such as Stanford where they are supposed to know better, go into the graduate engineering statistics class and have the students look at the data and draw conclusions from it and comment.  Allow them to propose measurements if they wish, then interpret that data.  Let them work together in groups if they want, or work as individuals.  Deal out a couple K of Monopoly money, and allow them to propose tests, so you simulate the real world where you can do expensive additional tests. 

Even with sharp students at their prime, I fear we would be appalled at how many draw the wrong conclusions.  I can imagine we would see some magic "95%" criterion show up in plenty of the reports, and whenever used, it would be misleading or incorrectly applied.  We might see it is better to know nothing about statistics than to know just enough to use statistics to support a wrong conclusion.

spike