Pater Familias

Theories, observations, and articles, by Philip Schaefer · All posts · Categories

WHY ARE ALMOST ONE THIRD OF ARTICLES IN SCIENTIFIC JOURNALS WRONG?

October 20, 2008 at 12:03 PM · Science

WHY ARE ALMOST ONE THIRD OF ARTICLES IN SCIENTIFIC JOURNALS WRONG? This article in the Economist for October 11 reports on an article by John Ioannidis and others which advances explanations for why so many published scientific articles turn out to be wrong. Dr. Ioannidis did a study three years ago of 49 papers in leading journals that had each been cited by more than 1000 other scientists. Within a few years almost one third of the papers had been refuted by later studies. Why does this happen? I posted previously on an article in the Economist subtitled “Why so much medical research is rot”, which pointed out that many articles are based on a misunderstanding of statistics. (Roughly, if I test 1000 hypotheses, on average I will get fifty results that I can claim are statistically significant–that I can say were “less than 5% likely to have come about by chance.”) Among the other explanations that Dr. Ioannidis and his colleagues advance are a bias toward publishing positive results and the “Winner’s Curse.”

Referenced in: WE KNOW LESS THAN COMPUTER MODELS SAY WE DO.; IN PRAISE OF NON-RANDOMIZED DATA GATHERING.; SMALL SAMPLES—ANOTHER PROBLEM WITH PUBLISHED ARTICLES.