MINRES

MINRES, or Minimum Residual method, is an iterative technique for solving linear systems of the form Ax = b where A is Hermitian (symmetric in the real case). It seeks approximate solutions x_k within the k-th Krylov subspace K_k(A, r0) that minimize the Euclidean norm of the residual r_k = b − A x_k.

The method builds an orthonormal basis for the Krylov subspace using the Lanczos process, yielding a small tridiagonal matrix T_k that captures A’s action in that subspace. At iteration k, the algorithm reduces the original problem to a least-squares problem min_y ||β e1 − T_k y||_2, where β is the norm of the initial residual and e1 is the first coordinate vector. The approximate solution is then x_k = x_0 + V_k y, with V_k containing the Lanczos vectors.

MINRES is particularly suited to symmetric matrices that are indefinite or singular, since it minimizes the residual without requiring positive definiteness. It can converge even when A has negative or zero eigenvalues, unlike some methods that require SPD matrices. Numerical breakdowns in the Lanczos process can occur, but these are typically handled by look-ahead strategies or minor modifications, and the method often remains robust for practical sparse problems.

In practice, each iteration involves one matrix-vector product with A and a small, inexpensive least-squares solve; memory usage grows with the number of stored Lanczos vectors. Consequently, per-iteration cost is modest for sparse A, making MINRES a popular choice for large symmetric systems. Related methods include CG for symmetric positive-definite A and SYMMLQ for SPD problems; GMRES covers general non-symmetric systems. A variant known as MINRES-QLP improves robustness for singular or ill-conditioned cases by producing a minimum-length solution.