multipletest

Multipletest is a term that generally refers to the problem of performing multiple statistical tests simultaneously. When many hypotheses are tested at once, the probability of making at least one Type I error (falsely rejecting a true null hypothesis) increases with the number of tests performed. This phenomenon is often referred to as the "multiplicity problem" or "multiple comparisons problem."

To address this, various methods have been developed to adjust the significance level of individual tests or to control the overall error rate across all tests. Common approaches include the Bonferroni correction, which divides the desired family-wise error rate (FWER) by the number of tests, and the False Discovery Rate (FDR) control methods, such as the Benjamini-Hochberg procedure, which aim to control the expected proportion of rejected null hypotheses that are actually true.

The choice of method depends on the research context and the specific goals of the analysis. For example, controlling the FWER is more conservative and suitable when the cost of any false positive is high, while controlling the FDR is often preferred in exploratory research where a higher number of false positives might be acceptable in exchange for greater power to detect true effects. The need for multiple testing correction is particularly relevant in fields like genomics, neuroimaging, and high-throughput screening where thousands or millions of tests may be conducted.