exponentialfamily

An exponential family is a broad class of probability distributions that can be written in a standardized form that highlights its sufficient statistics and natural parameters. A density (or mass function) belongs to the exponential family if it can be expressed as f(x|η) = h(x) exp( η^T T(x) − A(η) ), where η is the natural parameter, T(x) is a vector of sufficient statistics, h(x) is the base measure, and A(η) is the log-partition function that ensures normalization.

Key components and properties: the natural parameterization separates data-dependent terms (through T(x) and h(x)) from the parameter-dependent normalization A(η). The log-partition function A(η) is convex, and moments are obtained by derivatives: E[T(X)] = ∇A(η), Var(T(X)) = ∇^2A(η). In regular exponential families, the support of x does not depend on η; if the support depends on η, the family is curved or non-regular, with different theoretical implications.

Examples and scope: common members include the Poisson distribution, which can be written as f(x|λ) = exp(x log λ − λ − log x!), with η = log λ. The Bernoulli distribution also fits, via f(x|p) = exp(x log(p/(1−p)) + log(1−p)), with η = log(p/(1−p)). The normal distribution with known variance can be cast into exponential form by choosing appropriate T(x) and η. The exponential and gamma families are closely related within this framework, illustrating the breadth of the class.

Applications: exponential families underpin many statistical methods, including generalized linear models, where the mean is linked to linear predictors through the natural parameter, and Bayesian analysis, where conjugate priors align with exponential-family likelihoods.