GMRES

GMRES, or Generalized Minimal Residual method, is an iterative method for solving nonsymmetric or non-Hermitian linear systems Ax = b. It builds successive approximations by expanding in a Krylov subspace and choosing the approximate solution that minimizes the Euclidean norm of the residual over that subspace.

Starting from an initial guess x0 with residual r0 = b − Ax0, GMRES constructs an orthonormal basis V1, V2, ..., Vm for the Krylov subspace Km(A, r0) via the Arnoldi process. This yields an upper Hessenberg matrix Hm such that AVm = Vm+1Hm+1,m. The approximate solution x m is x0 plus Vm y, where y solves the least-squares problem min ||βe1 − Hm y||2, with β = ||r0||2. The residual norm can be updated efficiently during the iteration using Givens rotations, avoiding explicit computation of y until the end of the cycle.

A common variant is GMRES(m), or restarted GMRES, which stops after m iterations, uses the current x m as the new starting point, and restarts to control memory usage. Convergence can be robust for nonsymmetric problems, but memory and compute grow with m, and convergence may stall under restart.

GMRES relies on preconditioning to accelerate convergence. Left, right, or split preconditioning with ILU, Jacobi, or block Jacobi schemes is typical. Variants such as Flexible GMRES (FGMRES) accommodate changing preconditioners; GMRES-DR (deflated/restarted) and other deflated or restarted approaches improve performance by incorporating approximate eigenvectors.

GMRES was introduced by Youcef Saad and Martin H. Schultz in 1986 as a robust Krylov-subspace method for general linear systems.