NHST

Null Hypothesis Significance Testing (NHST) is a statistical framework used to assess whether observed data provide enough evidence to reject a null hypothesis. It combines the calculation of p-values with predefined decision rules to determine statistical significance.

In NHST, researchers specify a null hypothesis H0 and an alternative H1, choose a significance level alpha (commonly 0.05), and compute a test statistic under H0. The p-value is the probability, assuming H0 is true, of obtaining results at least as extreme as those observed. If the p-value is less than or equal to alpha, H0 is rejected; otherwise, results are not considered statistically significant.

Key concepts include Type I error (probability alpha, rejecting a true H0) and Type II error (failing to reject a false H0) with power equal to 1 - beta. NHST emphasizes long-run error control but does not provide the probability that H0 is true given the data.

Limitations include misinterpretation of p-values, reliance on arbitrary thresholds, sensitivity to sample size, and neglect of practical significance. Multiple comparisons and publication bias can distort results. Best practices advocate reporting effect sizes, confidence intervals, preregistration, and, where appropriate, alternative methods.

History: NHST draws on Fisher's p-value concept and Neyman–Pearson decision theory. It remains widespread across disciplines but has faced criticism for encouraging binary decisions. Debates have increased calls to supplement or replace NHST with estimation-focused approaches, Bayesian methods, and better reporting standards.