opticalflow

Optical flow is the pattern of apparent motion of brightness patterns in an image sequence, caused by relative motion between the observer and the scene. It is commonly represented as a dense 2D vector field where each vector indicates the displacement of a pixel from one frame to the next. The concept underlies motion estimation in computer vision and video processing.

The problem dates to classic work by Horn and Schunck (1981) introducing a global energy model with brightness constancy and smoothness constraints. Earlier local methods include Lucas and Kanade (1981) which estimate motion from a small neighborhood. The aperture problem—ambiguity in motion direction within small windows—motivates regularization and multi-scale approaches.

Dense optical flow computes a flow vector for every pixel, while sparse methods track a subset of features. Common strategies combine data terms enforcing brightness constancy with regularization promoting smoothness; pyramidal or coarse-to-fine schemes help capture large motions.

Recently, deep learning models such as FlowNet, PWC-Net, and RAFT achieve high accuracy by learning to predict flow fields from image pairs, often using unsupervised or self-supervised losses and handling occlusions.

Applications span video compression, object tracking, autonomous navigation, robotics, and visual odometry. Limitations include sensitivity to illumination changes, occlusion, large displacements, and non-rigid motion. Evaluation uses endpoints error and angular error on benchmark datasets like KITTI, Middlebury, and Sintel.