permutationstest

A permutation test, also called a randomization test, is a nonparametric method for testing hypotheses by using the data themselves to build the null distribution. It relies on the idea that, under the null hypothesis, the labels or assignments of observations to groups are exchangeable, so all labelings are equally likely.

The basic procedure is straightforward. Choose a test statistic that measures the effect of interest, such as the difference in means between two groups or a statistic based on ranks. Compute the observed value of this statistic. Then repeatedly permute the data labels (for example, shuffle group membership) and, for each permutation, recompute the statistic. The collection of permuted statistics forms the null distribution. The p-value is the proportion of permutations that yield a statistic as extreme as or more extreme than the observed one. For a two-sided test, both tails are considered; for a one-sided test, only the relevant tail is used.

Common variants include the two-sample permutation test, used to compare independent groups; the paired or matched-pairs permutation test, used for data with a natural pairing; and the one-sample permutation test, used to compare a sample to a reference value by permuting signs or residuals. When all possible permutations are feasible, the test is exact; otherwise, Monte Carlo sampling provides an approximation.

Permutation tests are distribution-free and robust to many deviations from parametric assumptions, making them suitable for small samples or nonnormal data. Their main limitations are computational intensity for large datasets and sensitivity to the chosen test statistic. They are widely used in clinical trials, ecology, genomics, and other fields seeking distribution-free inference.