CartPole

The cart-pole system is a classic control problem in which a cart can move horizontally on a track while carrying a pendulum mounted on the cart. The objective is to balance the pendulum in the upright position by applying a horizontal force to the cart. The problem is inherently nonlinear and underactuated, making stabilization challenging.

Mathematically, the model consists of a cart of mass M on a frictionless track and a pendulum of mass m and length l attached to the cart. Gravity is g, and the control input is a horizontal force u applied to the cart. Let x be the cart position and theta the pendulum angle from the vertical, with theta = 0 corresponding to upright. The nonlinear equations of motion are commonly written as:

(M + m) x'' + m l theta'' cos theta − m l (theta')^2 sin theta = u

l theta'' + x'' cos theta + g sin theta = 0

The system state is typically x = [x, x_dot, theta, theta_dot].

Control methods often start with linearization around the upright equilibrium for small deviations, yielding design approaches such as linear quadratic regulators. More generally, nonlinear model-based controls, model predictive control, and reinforcement learning techniques can handle larger deviations and model uncertainties. The problem is widely used as a testbed for balancing and policy optimization methods.

In practice, the cart-pole appears in AI and control benchmarks. In OpenAI Gym, the CartPole environment uses a discrete action space (push left or right) with a reward of +1 per time step while the pole remains within bounds. Observations typically include [x, x_dot, theta, theta_dot], and the task ends when the angle or cart position exceeds thresholds.