pcurve

P-curve is a statistical tool used to assess the evidential value of a set of empirical studies by examining the distribution of its significant p-values. Introduced by Jonathon Simonsohn, Uri Simonsohn, and Leif Nelson and Simmons in 2014, p-curve is designed to help researchers distinguish genuine effects from selective reporting or p-hacking within a literature body.

The method collects all significant p-values (typically those less than 0.05) from independent tests of the same hypothesis. The shape of the p-curve conveys information about the underlying effect and study power. When there is a true effect and reasonable power, the p-curve tends to be right-skewed, with more very small p-values. If there is no true effect, the p-curve is expected to be roughly uniform or left-skewed among the significant p-values. A conspicuous excess of p-values just below 0.05 can indicate p-hacking or selective reporting.

Statistical procedures associated with p-curve include tests for skewness, as well as binomial or chi-square tests comparing observed counts of p-values in predefined intervals (for example, 0–0.01, 0.01–0.02, 0.02–0.05) to expectations under different models. P-curve analyses can also inform estimates of average power and potential effect sizes across studies, though they rely on assumptions such as independence of tests and accurate reporting of p-values.

Limitations include sensitivity to selection bias, multiple testing within studies, and heterogeneity among studies. Critics note that p-curve is not a definitive test of a single hypothesis and should be used alongside broader methodological assessments and pre-registration practices.