[ExI] Paper on costly errors in the scientific research and publication process

[The Avantguardian]

>
>http://knowledge.wharton.upenn.edu/article.cfm?articleid=3029 
>
>
Relevant excerpt:
-------------------------------------------
While Simonsohn acknowledges that academics have long been concerned about the liberties that some researchers take when analyzing data, he highlights three unique contributions that his paper makes. First, he and his co-authors offer a simple, low-cost solution to the problem that does not interfere with the work of scientists already doing everything right -- asking them to disclose what they did in ways that would add only a few dozen words to most articles. Second, they demonstrate just how big the problem can get: While up to now, it was suspected that the consequences of taking these liberties were relatively minor, the authors show it can increase the odds of finding evidence for a false hypothesis to over 50%. And third, the authors did an actual experiment to illustrate their point about how data are manipulated to achieve a desired outcome. 
---------------------------------------
 
 
Interesting article, Max. It seems psychology research is suffering from problems similar to those of medical research as described by this article first brought to our attention by BillK a few weeks ago:
 
http://health.yahoo.net/news/s/nm/in-cancer-science-many-discoveries-don-t-hold-up

-------------------
During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 "landmark" publications -- papers in top journals, from reputable labs -- for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development.
Result: 47 of the 53 could not be replicated. He described his findings in a commentary piece published on Wednesday in the journal Nature.
"It was shocking," said Begley, now senior vice president of privately held biotechnology company TetraLogic, which develops cancer drugs. "These are the studies the pharmaceutical industry relies on to identify new targets for drug development. But if you're going to place a $1 million or $2 million or $5 million bet on an observation, you need to be sure it's true. As we tried to reproduce these papers we became convinced you can't take anything at face value."
--------------------
 
I am sure more than one scientific career has gone down in flames trying to replicate a false positive reported in a prestigious journal. The life-sciences and social sciences really need to fix this problem or religious people will feel justified in disbelieving science. Since religious people vote, that of course means less funding for science which makes it even more competitive and prone to fraud. It's like a nasty chain reaction that's turning the well into poison.
  
Stuart LaForge

"Man is a strange animal, he doesn't like to read the handwriting on the wall until his back is up against it."  -Adlai Stevenson