likelihoods

Likelihoods refer to the likelihood function in statistics, a function that measures how probable the observed data are for different values of unknown parameters. Given a statistical model and data, the likelihood is a function of the parameter(s) with the data treated as fixed. It contrasts with probability, which treats the parameter as fixed and the data as random. Formally, if X has density f(x; θ) under parameter θ, the likelihood is L(θ) = f(X; θ).

A common use is maximum likelihood estimation (MLE): the parameter value that maximizes the likelihood is chosen as the point estimate. The log-likelihood, l(θ) = log L(θ), is often used for numerical stability and easier calculus. For example, with a Binomial(n, p) model and k observed successes, the likelihood is L(p) = p^k (1−p)^{n−k}, and the MLE is p̂ = k/n.

Likelihoods also support more complex inference. Likelihood ratios compare how well different parameter values explain the data, with the likelihood ratio test statistic defined as 2[log L(θ1) − log L(θ0)]. The profile likelihood fixes nuisance parameters before maximizing over the parameter of interest. In Bayesian inference, the likelihood combines with a prior to form the posterior distribution via Bayes’ theorem.

Key properties include that the likelihood is not a probability distribution over θ and that, under regularity conditions, the MLE is invariant under smooth reparameterizations. The likelihood principle states that all evidence about θ from the data is contained in the likelihood function.