NMSLIB

NMSLIB, short for Non-Metric Space Library, is an open-source library for efficient similarity search and approximate nearest neighbor (ANN) search in high-dimensional spaces, with a focus on non-metric and heterogeneous spaces. It is designed to scale to large datasets and supports a variety of distance measures and data types.

The library uses a modular architecture based on the concepts of spaces and indexes. A Space defines how distance or similarity is computed between items, while an Index implements an ANN algorithm. NMSLIB provides multiple index implementations, most notably the Hierarchical Navigable Small World (HNSW) method, along with other graph-based and partitioning approaches. This combination allows users to tailor the search structure to their data characteristics and performance requirements.

NMSLIB is implemented in C++ with bindings for Python and other languages. It offers a programmatic API for creating spaces and indexes, building indexes from data, and performing nearest-neighbor queries. The toolkit supports multi-threaded indexing and querying, as well as a range of distance metrics suitable for real-valued vectors, binary data, and non-metric spaces.

Applications for NMSLIB include information retrieval, recommendation systems, image and text similarity, and other domains that require fast similarity search over large collections. It is used in both research and production environments to enable scalable similarity-based retrieval and benchmarking across different spaces and metrics.

NMSLIB is available as open-source software hosted on public repositories. It is distributed under a permissive open-source license, with documentation and tutorials provided to guide installation, usage, and integration into applications.