maximalelikelihood

Maximum likelihood estimation (MLE) is a method for estimating the parameters of a statistical model by maximizing the likelihood function, which represents the probability of observing the given data under different parameter values. If a sample x1, x2, ..., xn is drawn from a distribution with density or mass function f(x; θ), the likelihood is L(θ) = ∏i f(xi; θ), and the goal is to find θ̂ that maximizes L. Because logarithms preserve the order of values, it is common to maximize the log-likelihood ℓ(θ) = log L(θ) = Σi log f(xi; θ).

MLE has several key properties under regularity conditions. The estimator θ̂ is consistent, meaning it converges in probability to the true parameter θ0 as the sample size grows. It is typically asymptotically normal, with a distribution centered at θ0 and variance equal to the inverse of the Fisher information, I(θ0). The method is asymptotically efficient among regular estimators, achieving the Cramér–Rao lower bound in large samples. An important invariance property states that if θ̂ maximizes L, then any smooth function g(θ̂) maximizes L̃ for the transformed parameter φ = g(θ).

Computationally, closed-form solutions exist for simple models, but many problems require numerical optimization (e.g., Newton–Raphson, gradient ascent, BFGS) or specialized algorithms like the expectation-maximization (EM) algorithm for latent-variable models.

Common considerations include model misspecification, boundary estimates, identifiability issues, and small-sample bias. In practice, MLE is often complemented by Bayesian methods (e.g., MAP estimation) or resampling techniques to assess uncertainty.