Mestimator

Mestimator (M-estimator) is a broad class of estimators in statistics defined as the minimizer of a sum of a function rho of residuals, or equivalently as the solution to an estimating equation based on a score function psi. The “M” often stands for maximum likelihood-type or minimum contrast, reflecting that many M-estimators resemble maximum likelihood estimators but use a different loss function.

In a typical setting, suppose data consist of independent observations (x_i, y_i) and a model y_i ≈ f(x_i, theta). An M-estimator theta_hat solves

min_theta sum_i rho(y_i - f(x_i, theta)).

Equivalently, it satisfies the estimating equations sum_i psi(y_i - f(x_i, theta)) ∂f/∂theta (x_i, theta) = 0, where psi = rho' is the derivative of rho with respect to its argument. For a location model y_i - theta, this reduces to sum_i psi(y_i - theta) = 0.

Examples and robustness: If rho(u) = u^2, psi(u) = 2u, and theta_hat is the ordinary least squares estimator. Robust choices of rho, such as Huber’s rho or Tukey’s biweight, yield M-estimators with bounded or redescending influence, making them less sensitive to outliers in the data.

Properties: Under suitable regularity, M-estimators are consistent and asymptotically normal. The asymptotic variance has the form A^{-1} B A^{-1}, where A = E[psi'(Z)], B = E[psi(Z)^2] for centered residual Z. The robustness of an estimator is reflected by the influence function of theta_hat; bounded psi leads to bounded influence. The breakdown point and efficiency depend on the chosen rho and the model; robust variants such as S- and MM-estimators aim to improve these aspects.

Computation often uses iterative reweighted least squares (IRLS) or related fixed-point algorithms. M-estimators encompass a wide range of robust location and regression methods and relate closely to maximum likelihood whenever rho corresponds to a log-likelihood up to a constant.