exchangeability

Exchangeability is a property of a sequence of random variables X1, X2, ..., such that for every finite n and every permutation π of {1, ..., n}, the joint distribution of (X1, ..., Xn) equals that of (Xπ(1), ..., Xπ(n)). Intuitively, the order of observations conveys no information about the joint distribution; only the multiset of observed values matters.

Exchangeability does not imply independence. While independent sequences are exchangeable, the converse is not necessarily true. A canonical example is draws from a finite population without replacement: the joint distribution of the first n draws is invariant under permutations, but the variables are not independent. A constructive model is Xi given a latent parameter p are i.i.d. Bernoulli(p); marginally, the Xi are exchangeable.

Infinite exchangeability is particularly amenable to representation theorems. de Finetti’s theorem states that any infinite exchangeable sequence can be represented as conditionally independent and identically distributed given a latent parameter (or directing random measure). In the binary case, this means the Xi are i.i.d. Bernoulli(p) given p, with p having some prior distribution (Beta in the canonical case).

Applications and implications. In Bayesian statistics, exchangeability justifies modeling repeated observations as arising from a mixture of i.i.d. processes and supports hierarchical modeling and the use of nonparametric priors such as Dirichlet processes. It explains why observed frequencies can converge to a latent distribution when the exchangeability assumption is plausible.

Limitations. Exchangeability is an idealization and may be violated by time trends, batch effects, or heterogeneity. Finite samples may only approximate exchangeability, and mis-specifying it can lead to misleading inferences. Assessing plausibility of exchangeability is important in applied work.