reparameterization

Reparameterization is a technique in statistics, mathematics, and machine learning in which a model or variable is expressed in terms of alternative parameters related by a deterministic transformation. The aim is to simplify estimation, inference, or optimization, improve numerical stability, or enable efficient sampling and gradient-based methods. A reparameterization is usually one-to-one so that the original quantities can be recovered from the new parameters, often with an explicit Jacobian factor when probability densities are transformed.

In statistics and Bayesian inference, reparameterization often involves moving to a more convenient parameter space, such as transforming a positive parameter by a logarithm or using a logit transform to bound a probability between 0 and 1. When densities are changed, the Jacobian determinant is used to adjust the probability mass or density to preserve correct probabilities under the new parameterization.

In machine learning, the reparameterization trick is a central application, enabling gradient-based optimization through stochastic nodes. For example, in variational autoencoders, a latent variable z is expressed as z = μ + σ ⊙ ε with ε ~ N(0, I). This reformulates sampling as a deterministic function of ε and the parameters μ, σ, allowing backpropagation to update those parameters. Variants include continuous relaxations such as the Gumbel-Softmax for discrete choices and continuous normalizing flows that reparameterize complex distributions via invertible transformations.

In geometry and numerical analysis, reparameterization also refers to changing the parameterization of a curve or surface to simplify computations, such as arc-length parameterization or aligning with a particular coordinate system.