hypothesetests

Hypothesis testing is a statistical method used to assess evidence about a population parameter based on sample data. It formalizes the process of making inferences by comparing observed data to expectations under a null hypothesis, typically representing no effect or no difference.

In a hypothesis test, researchers specify a null hypothesis (H0) and an alternative hypothesis (H1). A significance level, often denoted alpha (such as 0.05), sets the threshold for evidence required to reject H0. A test statistic is computed from the data, and a p-value is derived, representing the probability of observing data at least as extreme as the sample under H0. If the p-value is at most alpha, H0 is rejected; otherwise it is not rejected. Many reports use the phrasing "reject H0" or "fail to reject H0" rather than declaring H0 true or false. Tests can be one-tailed or two-tailed depending on whether the alternative specifies a direction of effect.

Common tests include z-tests and t-tests for means, chi-square tests for categorical data, and ANOVA for comparing more than two means. Nonparametric alternatives exist for situations with nonnormal data or small samples (for example, Mann–Whitney, Kruskal–Wallis). Assumptions about randomness, independence, and distribution shape influence test validity and may guide the choice of test or its robustness.

Power, the probability of correctly rejecting a false H0, is a key consideration, along with the risks of Type I error (false positives) and Type II error (false negatives). Multiple testing can inflate false positives, prompting adjustments. Hypothesis testing complements confidence intervals, with many two-sided intervals corresponding to standard significance levels.