July 9th, 2024

Ranking Fields by p-Value Suspiciousness

The article addresses p-hacking in research, focusing on suspicious p-value clustering across fields. Economics shows more credible results. Replication crisis is universal, emphasizing the call for research integrity and transparency.

Read original articleLink Icon
Ranking Fields by p-Value Suspiciousness

The article discusses the issue of p-hacking in scientific research, where researchers manipulate data to achieve statistically significant results. The focus is on the distribution of p-values in various fields, with an emphasis on the "uncanny mountain" of p-values between 0.10 and 0.01, suggesting potential data manipulation. The analysis reveals that many fields exhibit suspicious clustering of p-values, indicating possible bias or selective reporting. Economics stands out as having relatively more credible results compared to other disciplines, attributed to rigorous reporting practices and a focus on statistical robustness. The replication crisis is highlighted as a universal challenge across all fields due to p-hacking and questionable research practices. The study examines large datasets from diverse fields to assess the prevalence of suspicious p-values and emphasizes the need for improved research integrity and transparency to address these issues.

Related

The case for criminalizing scientific misconduct · Chris Said

The case for criminalizing scientific misconduct · Chris Said

The article argues for criminalizing scientific misconduct, citing cases like Sylvain Lesné's fake research. It proposes Danish-style committees and federal laws to address misconduct effectively, emphasizing accountability and public trust protection.

Hack Your Way to Scientific Glory

Hack Your Way to Scientific Glory

FiveThirtyEight offers a guide for social scientists analyzing political parties' impact on the U.S. economy since 1948. Real data, statistical significance, key factors, influential positions, and monthly plotting are recommended. Scholars and data sources are referenced.

Latest Breakthrough from String Theory

Latest Breakthrough from String Theory

Researchers introduced a new series from string theory to enhance pi extraction for quantum calculations. Despite media hype, doubts persist about its novelty and practical benefits, urging careful evaluation amid exaggerated coverage.

Journal retracts all 23 articles in special issue

Journal retracts all 23 articles in special issue

A journal retracted 23 articles from a special issue due to compromised peer review. Guest editor Abbas Mardani didn't comment. Authors criticized lack of transparency and faced consequences. Publisher Springer mentioned ongoing investigations.

Research into homeopathy: data falsification, fabrication and manipulation

Research into homeopathy: data falsification, fabrication and manipulation

Research on homeopathy faces credibility issues due to data manipulation in a study led by Michael Frass. The study, once positive, now raises concerns of scientific misconduct, urging withdrawal of publication. Challenges persist in alternative medicine research, highlighting the conflict between ideology and scientific integrity.

Link Icon 2 comments
By @Rastonbury - 5 months
Does this analysis remove the effect of publication bias? It is not clear to me how the author did this although they do call it out
By @readthenotes1 - 5 months
Economics seems to be as powerful as astrology, other fields less so...