Loglike

Loglike, short for log-likelihood, is the natural logarithm of the likelihood function used in statistics. Given a statistical model with parameter vector theta and observed data x, the likelihood L(theta) = p_theta(x) expresses the probability of observing x under theta. The log-likelihood l(theta) = log L(theta) is often preferred because it converts products into sums for independent observations, improves numerical stability, and is differentiable, facilitating optimization.

For independent data x1, ..., xn from the model, l(theta) = sum_i log p_theta(x_i). The maximum likelihood estimate (MLE) is the theta that maximizes l(theta).

Examples: A Gaussian model with mean mu and known variance sigma^2 has l(mu) = -n/2 log(2π sigma^2) - (1/(2 sigma^2)) sum (x_i - mu)^2. A Bernoulli model with success probability p has l(p) = sum_i [x_i log p + (1 - x_i) log(1 - p)].

Log-likelihood is central to statistical inference. It underpins likelihood ratio tests, information criteria (AIC, BIC), and serves as the basis for gradient-based optimization in both classical statistics and many machine learning methods. Note that the log-likelihood is not itself a probability; it is a function of the model parameters given the data.

Numerical issues include non-convexity and flat regions; regularization or reparameterization can help. The log-likelihood is differentiable under regularity conditions, with the gradient equal to the sum of score functions and the Hessian providing curvature near the optimum.