likelihoodestimation

Likelihood estimation is a method of parameter estimation that uses the likelihood function L(θ|x) = ∏ f(x_i|θ) for a parametric model. The likelihood is viewed as a function of θ given the observed data x, not of the data given θ. The aim is to identify parameter values that maximize L.

Maximum likelihood estimation (MLE) is the standard approach: θ_hat = argmax_θ L(θ|x). In practice log-likelihood l(θ)=log L is maximized. Under regularity conditions, MLEs are consistent, asymptotically normal, and efficient, achieving the Cramér-Rao lower bound.

Invariance and examples: MLEs are invariant under reparameterization: if φ=g(θ), then φ_hat=g(θ_hat). A simple example: coin toss with data k heads in n trials; L(p)=p^k(1-p)^{n-k}; the MLE p_hat = k/n.

Extensions and computation: for incomplete or latent data, the EM algorithm iteratively increases the likelihood. Profile likelihood handles nuisance parameters. Likelihood ratio tests compare nested models using the distribution of -2 log λ.

Relationship to Bayesian methods and model selection: in Bayesian inference, the likelihood is combined with a prior to form a posterior; the likelihood itself is central to both paradigms. Compare models using likelihood-based criteria such as AIC or BIC that incorporate L and a complexity penalty.