ExpectationMaximizationVerfahren

The Expectation-Maximization algorithm, commonly abbreviated as EM, is an iterative method for finding maximum likelihood estimates of parameters in statistical models that involve latent variables or incomplete data. It alternates between estimating missing data given the current parameters (the E-step) and optimizing the parameters given this estimate (the M-step).

In the E-step, the algorithm computes the expected value of the log-likelihood of the complete data with respect to the current parameter estimates. This effectively imputes or accounts for the missing data by their conditional expectation conditioned on the observed data. In the M-step, it maximizes this expected log-likelihood with respect to the model parameters, yielding updated estimates. Each iteration is designed to increase, or at least not decrease, the observed-data likelihood, providing a monotonic convergence property.

EM is widely used in situations with latent structure, such as mixture models where component membership is unknown, hidden Markov models, and factor analysis. It provides a principled framework for handling missing or hidden information and often yields closed-form update rules for common models. Variants include generalized EM, which requires only an increase in the likelihood, as well as stochastic EM and online EM for large-scale data.

Practical considerations include the sensitivity to initialization, the choice of convergence criteria, and the possibility of converging to local maxima or saddle points. Extensions address issues of identifiability and singularities, and regularization can be incorporated to improve robustness. EM was introduced by Dempster, Laird, and Rubin in 1977 and remains a foundational tool in statistics and machine learning.