Reweighting

Reweighting is a statistical technique used to adjust the influence of observations so that analyses reflect a target distribution different from the one from which the data were drawn. It is commonly employed when collecting data from a population with a known but different distribution, or when models learned on one distribution must be applied to another.

The core idea is to assign a weight to each observation, typically proportional to the ratio of the target distribution to the source distribution at the observed value. In importance sampling, the weight for a sample x is w(x) = p_target(x) / p_source(x), and expectations under the target distribution are estimated as weighted averages of functions of the samples. In causal inference and survey sampling, inverse probability weighting uses weights like 1/P(T|X) or 1/e(X) to correct for nonrandom treatment assignment or sampling bias.

Applications span several fields. In machine learning and statistics, reweighting addresses covariate shift and class imbalance by emphasizing underrepresented cases or by aligning training and test distributions. In causal inference, inverse probability weighting helps estimate causal effects from observational data. In survey methodology, post-stratification and calibration weighting adjust survey samples to match known population margins. In physics and computational science, histogram or event reweighting updates results when model parameters change, avoiding full re-simulation.

Key considerations include choosing an appropriate target distribution, ensuring sufficient overlap between distributions, and managing high-variance weights. Normalization and weight stabilization techniques are common to improve estimator reliability. Overly large or unstable weights can lead to biased or imprecise results, especially when the target and source distributions differ substantially.