blockcoordinate

Blockcoordinate, often referred to as block coordinate descent (BCD) or block-wise optimization, is an iterative method for minimizing a function by updating blocks of variables while keeping the remaining blocks fixed. The variable vector x is partitioned into m blocks x = (x1, x2, ..., xm), with each xi belonging to its own domain Xi. At each iteration, one or more blocks are updated by solving a subproblem that minimizes the objective with respect to that block, given the current values of the other blocks.

In the standard cyclic scheme, at iteration t and block k, the update is xk := argmin over xk in Xk of f(x1^{t+1}, ..., x_{k-1}^{t+1}, x_k, x_{k+1}^t, ..., x_m^t). Variants include random or greedy block selection, and the use of proximal terms or regularization to handle non-smooth or constrained components. When the objective includes non-differentiable terms, proximal or accelerated proximal updates are common. For smooth components, block gradient steps or block coordinate gradient descent are used.

Convergence properties depend on problem structure. If f is convex in every block and the feasible set is convex, BCD typically produces a nonincreasing sequence with convergence to a global minimizer or a stationary point under mild conditions. For nonconvex problems, BCD often converges to a stationary point, with convergence guarantees requiring additional assumptions such as block-wise Lipschitz continuity and appropriate update rules.

Blockcoordinate methods are widely used in machine learning and statistics, including sparse regression (lasso and elastic net), matrix factorization, collaborative filtering, and high-dimensional optimization problems where full gradient methods are impractical. They offer simplicity, scalability, and the ability to exploit separability in objectives and constraints.