Fishervectors

Fishervectors are a powerful image representation technique used in computer vision and machine learning. They extend the concept of bag-of-visual-words (BoVW) models by incorporating gradient information from local feature descriptors. Instead of simply counting occurrences of visual words, Fishervectors represent images as points in a high-dimensional space defined by the gradients of a generative model, typically a Gaussian Mixture Model (GMM).

The process begins by extracting local features, such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features), from an image. These features are then clustered to form a visual vocabulary. For each local feature in an image, its deviation from the mean of its assigned visual word is calculated. These deviations, along with their variances, are then pooled together to form the Fishervector. This pooling effectively captures the "visual words" that are present in the image and how they are distorted or varied.

The resulting Fishervector is a dense, high-dimensional descriptor that encodes richer information than a simple BoVW histogram. This richness allows for more discriminative image representations, leading to improved performance in tasks like image classification, object recognition, and image retrieval. Fishervectors have been shown to outperform BoVW models significantly, especially in scenarios with large datasets and complex visual patterns. Their ability to capture fine-grained details and spatial configurations makes them a valuable tool in modern computer vision pipelines.