reparameterise

Reparameterisation, or reparameterization, is the process of expressing a model or optimization problem in terms of an alternative set of parameters that describes the same underlying object. The goal is to improve numerical stability, enforce constraints, or simplify estimation and inference.

Common techniques replace constrained parameters with unconstrained ones via invertible, differentiable transformations. Examples include using a logarithm for positive quantities, a logit or probit transform for probabilities, or a Cholesky decomposition for covariance matrices. Reparameterisation can reduce posterior correlation and make priors and likelihoods more tractable in both Bayesian inference and machine learning.

One widely used approach in stochastic models is the reparameterization trick. If a random variable X depends on parameters theta, one can represent X as a deterministic function of theta and an auxiliary random variable epsilon with a fixed distribution, such as X = g(theta, epsilon). This permits gradient-based optimization to flow through X, which is essential in variational autoencoders and related methods.

Care is required when transforming densities. The change of variables introduces a Jacobian term, and priors must be transformed accordingly. Reparameterisation can also affect interpretability and identifiability; not all transforms are suitable or beneficial.

Reparameterisation is a general concept across statistics, probability, and optimization, distinct from, but related to, standard parameterization choices in a model. See also parameterization, transformation, Jacobian, and the reparameterization trick.