Nulltests

Nulltests, or null hypothesis tests, refer to statistical procedures used to decide whether there is evidence against a null hypothesis H0 about a population parameter or distribution. A typical null test specifies H0, an alternative Ha, a chosen significance level alpha, and a test statistic derived from the sample data. The distribution of the statistic under H0 is used to compute a p-value or to define a critical region. If the observed statistic falls in the critical region or the p-value is at most alpha, H0 is rejected in favor of Ha; otherwise it is not rejected.

Common formal tests include the t-test for a mean, the z-test, the chi-square test for variance or goodness-of-fit, the F-test for multiple variances, and nonparametric or exact tests such as Fisher's exact test or the McNemar test. Permutation tests and bootstrap tests provide null distributions by resampling under the null hypothesis.

Key concepts include Type I error (rejecting H0 when it is true) controlled at level alpha; Type II error; power; one-tailed versus two-tailed tests; and how sample size and effect size influence power and detectability.

Nulltests are widely used in clinical trials, psychology, social sciences, and biological sciences. They are often contrasted with confidence intervals and Bayesian approaches. Criticisms include reliance on p-values and p-hacking; to mitigate, researchers use pre-registration, corrections for multiple testing, and emphasis on effect sizes and confidence intervals.

Variants of null testing include equivalence and non-inferiority tests, where the null expresses a meaningful difference and rejection supports similarity within a specified margin.