reparameterized

Reparameterized refers to expressing a parameter, model, or random variable in terms of an alternative set of parameters or an auxiliary source of randomness. The goal is often to simplify estimation, improve numerical stability, or enable differentiation and efficient sampling. Reparameterization can involve separating deterministic and stochastic components or transforming a parameter from a constrained space to an unconstrained one.

In machine learning, the term is closely associated with the reparameterization trick, notably used in variational autoencoders and related methods. A random variable z drawn from a parameterized distribution is represented as z = mu + sigma * epsilon, where epsilon is drawn from a fixed distribution (commonly standard normal). This form makes z differentiable with respect to mu and sigma, allowing gradient-based optimization through stochastic nodes.

Beyond learning tricks, reparameterization appears in statistical modeling to improve identifiability, exploration of the parameter space, or numerical stability. Transforming parameters, such as modeling a positive scale parameter as sigma = exp(eta), ensures positivity and can stabilize optimization. Location-scale transformations, X = mu + sigma Z with Z having a known distribution, are classic examples that facilitate sampling and inference.

Commonly cited examples include the Gaussian family, where sampling can be written as X = mu + sigma Z with Z ~ N(0,1). Reparameterization also underpins more complex constructions, such as hierarchical models and multivariate distributions, where covariance structures are parameterized via decompositions like Cholesky factors to enable efficient sampling and optimization.

Overall, reparameterization is a flexible concept used to recast models for better computation, inference, and interpretability across statistics and machine learning.