The past decade has seen numerous criticisms of statistical methodology in biomedical research, fuelled in part by a ‘reproducibility crisis’ – the concern that many supposedly significant results cannot be replicated. The latest salvo was an editorial recently published in the Journal of the American Medical Association (Ioannidis JPA. JAMA 2018;319:1429-1430).
“P values are misinterpreted, overtrusted, and misused,” states John Ioannidis, an epidemiologist at Stanford University. “Adopting lower P value thresholds [such as p<0.005] may help promote a reformed research agenda with fewer, larger, and more carefully conceived and designed studies.”
In 2005, Professor Ioannidis published a paper that stated that most published research findings are false (Ioannidis JPA. PLoS Med 2005;2:e124, free full text at www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/pdf/pmed.0020124.pdf). He also co-authored a survey of the biomedical literature, which found that the reporting of p-values increased from 7.3% of abstracts in 1990 to 15.6% in 2014 (33.0% in core clinical journals) (Chavalarias et al. JAMA 2016;315:1141-1148).