HorvitzThompsonEstimator

The Horvitz–Thompson estimator, named after Daniel N. Horvitz and Donovan J. Thompson, is a design-based method used in survey sampling to estimate finite population totals under unequal probability sampling. It was introduced in their 1952 work, A Generalization of Sampling Without Replacement. The estimator is unbiased for the population total when the inclusion probabilities are known and used correctly.

For a finite population U with units i each having a value y_i and inclusion probability π_i, the Horvitz–Thompson estimator of the total T = sum_{i in U} y_i is hat T_HT = sum_{i in sample} (y_i / π_i). If the population mean μ is desired and the population size N is known, a common form is hat μ_HT = hat T_HT / N, or equivalently hat μ_HT = (1/N) sum_{i in sample} (y_i / π_i). The estimator relies on the sampling design to provide the π_i values and, if needed, joint inclusion probabilities π_ij.

Variance and variance estimation: The design-based variance of hat T_HT is Var(hat T_HT) = sum_i sum_j (π_ij − π_i π_j) (y_i/π_i) (y_j/π_j), where π_ij are the joint inclusion probabilities. In practice, variance is estimated using design-based methods such as estimators that replace π_ij with known or estimated quantities, or via resampling techniques like jackknife or bootstrap tailored to the sampling design.

Applications and limitations: The Horvitz–Thompson estimator is widely used in official statistics and health and social surveys, especially with complex designs including unequal probabilities, PPS sampling, stratification, and clustering. Its main strength is unbiasedness under the correct design; its weaknesses include potentially high variance when weights are highly variable and reliance on accurate inclusion probabilities. Extensions include normalized or Hajek estimators and various variance estimation methods to improve efficiency.