PTests

PTests is a term used to describe a family of statistical hypothesis tests that rely on p-values to assess evidence against a null hypothesis. The concept encompasses a range of procedures, from traditional parametric tests to modern nonparametric and resampling-based approaches. In a typical PTest, researchers formulate a null hypothesis, choose an appropriate test statistic, determine the sampling distribution under the null (analytically or via resampling methods such as permutation or bootstrap), compute a p-value, and compare the result to a predefined significance level to decide whether to reject the null.

Common examples of tests that fall under the PTest umbrella include one-sample and two-sample t-tests for continuous data, z-tests when population variance is known, and chi-square tests for categorical data. Nonparametric equivalents like the Mann-Whitney U test and the Wilcoxon signed-rank test are also considered PTests when implemented with p-value calculations. In modern practice, PTests are frequently reported with effect sizes and confidence intervals to convey practical significance alongside statistical significance. Adjustments for multiple testing, such as Bonferroni correction or false discovery rate control, are commonly applied in PTests involving many comparisons.

Interpretation of PTests requires careful consideration of context, sample size, and study design. A small p-value indicates strong evidence against the null under the assumed model but does not measure the probability that the null is true or the practical importance of the result. Limitations include sensitivity to sample size, model assumptions, and the potential for misinterpretation when p-values are treated as the sole measure of evidence. See also hypothesis testing, p-value, confidence interval, and multiple testing.