costates

Costates, also known as adjoint variables or co-states, are auxiliary variables used in optimal control theory to quantify how the optimal value of a performance criterion changes with respect to the state of the system. They accompany the state variables and form part of the necessary conditions for optimality.

In a standard finite-horizon problem, consider a dynamical system ẋ = f(x,u,t) and a cost functional J = φ(x(T)) + ∫0^T L(x,u,t) dt. The Hamiltonian is H(x,u,λ,t) = L(x,u,t) + λ^T f(x,u,t). The costate λ(t) evolves according to the adjoint equation -λ̇ = ∂H/∂x, with terminal condition λ(T) = ∂φ/∂x(x(T)). The control u(t) is chosen to minimize the Hamiltonian at each time, typically satisfying ∂H/∂u = 0 (subject to any constraints). The resulting system forms a two-point boundary value problem.

Interpretation and properties: the costate λ(t) represents the sensitivity of the optimal cost-to-go to small changes in the state x(t); it is sometimes viewed as the shadow price of the state variable. In fixed-endpoint problems, costates are determined by integrating backward in time from the terminal condition. They generalize to infinite-horizon problems and to problems with constraints, where they play a role in dual formulations.

Computationally, costates are found alongside the state trajectory by solving the Hamiltonian system, often using shooting methods or collocation. In stochastic settings, the stochastic maximum principle introduces adjoint processes analogously. Costates are a fundamental concept linking dynamics, optimization, and economic interpretation of state-value sensitivities.