KanadeLucasTomasi

Kanade-Lucas-Tomasi, commonly referred to as the KLT feature tracker, is a method for tracking distinct image points across video frames. It is named for Takeo Kanade, Brian Lucas, and Takeo Tomasi, whose contributions in the 1980s and 1990s established the approach and popularized its use in computer vision.

The core idea of the KLT tracker is to estimate the motion of a small patch around each feature by applying the Lucas-Kanade optical flow framework. Under the assumption of brightness constancy and small motion, the displacement that best aligns the patch between successive frames is found by minimizing the sum of squared differences of pixel intensities. This leads to a linear least-squares system based on image gradients, which is solved iteratively to produce a displacement vector for each tracked feature.

In practice, the method is often implemented with a pyramidal, coarse-to-fine approach to handle larger motions and scale changes. The set of points chosen to track is typically selected using the Shi-Tomasi good-features-to-track criterion, which prefers corners with strong and stable gradient information, yielding robust features for tracking.

Applications of the KLT tracker span motion estimation, visual odometry, structure from motion, object tracking, and video stabilization. It is valued for its efficiency and simplicity, making it suitable for real-time systems. Limitations include sensitivity to illumination changes, occlusions, and non-rigid deformation, as well as potential drift or loss of tracked features over time if patches become unreliable.