EMalgoritm

EM algorithm, short for Expectation-Maximization, is an iterative method for maximum likelihood or maximum a posteriori estimation in statistical models that involve latent variables or missing data. The method alternates between two steps: the Expectation step and the Maximization step. In the E-step, with current parameter values θ^(t), it computes the expected value of the complete-data log-likelihood log p(X,Z | θ) with respect to the conditional distribution p(Z | X, θ^(t)). This yields a function Q(θ | θ^(t)). In the M-step, it maximizes Q with respect to θ to obtain updated parameters θ^(t+1).

The EM algorithm is grounded in the idea of treating latent variables Z as missing data and iteratively improving parameter estimates. Each iteration is designed so that the observed-data likelihood p(X | θ) does not decrease. Under regularity conditions, the sequence of estimates converges to a stationary point of the observed-data likelihood, which may be a local maximum or a saddle point. EM can be slow to converge near a solution and is sensitive to initialization because many models with latent structure have multiple local maxima.

EM requires that the complete-data likelihood be tractable to compute and maximize. When the E-step involves intractable integrals, variants such as Monte Carlo EM or variational EM are used. Common applications include Gaussian mixture models, hidden Markov models, and factor analysis, as well as data sets with missing entries where latent components are inferred during estimation.

Extensions of EM include hard EM (classification EM), stochastic EM, online EM, and Monte Carlo EM. The algorithm remains a foundational tool in probabilistic modeling and unsupervised learning for estimating parameters in the presence of latent structure.