pvalues

A p-value is a measure used in statistical hypothesis testing to quantify the strength of evidence against a null hypothesis. It is defined as the probability, under the assumption that the null hypothesis is true, of obtaining a test statistic at least as extreme as the one actually observed. A small p-value indicates that the observed data would be unlikely if the null hypothesis were correct.

In practice, the p-value is calculated within a specified statistical model and depends on the chosen test statistic, the sampling distribution, and the sampled data. It is compared to a pre-set significance level, alpha, to decide whether to reject the null hypothesis. If the p-value is less than alpha, the result is considered statistically significant at that level; otherwise, there is not enough evidence to reject the null.

P-values are commonly misinterpreted. They are not the probability that the null hypothesis is true, nor a direct measure of the size or importance of an effect. They do not indicate the probability that future results will be similar, nor do they correct for study design flaws or multiple comparisons. The interpretation of a p-value can be affected by sample size: very large samples can produce small p-values for tiny, practically meaningless effects, while small samples may yield non-significant p-values for substantial effects.

Related concepts include confidence intervals, which convey a range of plausible values for an effect size, and Bayesian alternatives, which quantify the probability of hypotheses given the data. Proper reporting often includes effect sizes, confidence intervals, and transparency about testing and corrections for multiple comparisons.