Likelihood

Likelihood is a fundamental concept in statistics that expresses how plausible different parameter values are given the observed data. It is a function of the parameters, not of the data themselves, and should be distinguished from probability, which assigns chances to outcomes given fixed parameters.

For a statistical model with data x, the likelihood L(θ; x) = P(x | θ) for discrete data, or f(x | θ) for continuous data. If a sample x1, ..., xn consists of independent observations from a distribution with density f(·|θ), then the likelihood is L(θ; x) = ∏_{i=1}^n f(x_i | θ). The likelihood is proportional to the joint probability of the observed data as a function of θ, but it is not itself a probability distribution over θ.

Inference based on likelihood often uses maximum likelihood estimation (MLE), which selects θ̂ that maximizes L(θ; x). Equivalently, one maximizes the log-likelihood for mathematical convenience. Under regularity conditions, θ̂ is consistent and asymptotically normal, with standard errors obtainable from the curvature of the log-likelihood or the Fisher information. The likelihood framework also supports methods such as profile likelihood for handling nuisance parameters and likelihood ratio tests for model comparison.

The likelihood principle states that all evidential content about θ in the data is contained in the likelihood function, and two data sets that yield proportional likelihoods lead to the same inferences about θ. This principle informs debates about statistical methods; Bayesian inference, for example, multiplies the likelihood by a prior to form a posterior distribution.

Examples include a coin toss with unknown p, where observing k heads in n tosses yields L(p) ∝ p^k (1−p)^{n−k}, and normal data with known variance, where the likelihood is maximized by the sample mean. The concept originated with Ronald A. Fisher and remains central to statistical practice.