importanceweighted

Importance weighting, sometimes written as importanceweighted, is a technique used in statistics and machine learning to correct for differences between the distribution that generated data and the distribution with respect to which predictions or expectations are desired. It relies on weighting each observed example by an importance weight that accounts for how likely the example would be under the target distribution compared with the sampling distribution. In a common form, if X is distributed according to a target distribution P and is drawn from a proposal distribution Q, the weight for a sample x is w(x) = p(x)/q(x). For any function f, the expectation under P can be estimated as E_P[f(X)] = E_Q[w(X) f(X)].

Applications include correcting covariate shift in supervised learning, where training and test inputs come from different distributions, and domain adaptation. In Monte Carlo methods, importance sampling uses weights to estimate integrals with respect to P using samples from Q. In reinforcement learning, off policy evaluation and learning employ importance weights to relate trajectories generated by a behavior policy to a target policy. In deep learning, importance weighting appears in architectures such as importance-weighted autoencoders (IWAE), which use multiple importance-weighted samples to tighten variational bounds.

Techniques to manage variance and degeneracy of weights include self-normalized importance weighting, where weights are normalized to sum to one, and weight clipping or resampling. Practical concerns include high variance when the proposal distribution poorly covers the target distribution and support mismatches where q(x) = 0 but p(x) > 0, which can lead to biased estimates.

Related concepts include Monte Carlo integration, covariate shift, off-policy evaluation, and domain adaptation. The approach is a foundational tool for reweighting data, enabling more accurate estimation and learning when direct sampling from the target distribution is difficult.