1NN

1NN, short for 1-nearest neighbor, is one of the simplest supervised learning algorithms. It can be used for both classification and regression. In classification tasks, the predicted label of a new instance is the label of the closest instance in the training data, according to a chosen distance metric. In regression, the predicted value is the value of the closest training instance.

Training and prediction: There is essentially no training phase beyond storing the dataset; the algorithm is lazy. Predictions require computing distances to all training samples (unless an indexing structure is used). Common distance metrics include Euclidean distance for continuous features, Manhattan or Minkowski distances, and cosine distance for high-dimensional or sparse data. Tie-breaking strategies may be needed when two or more training points are equally close.

Properties and limitations: Pros include simplicity, no explicit training phase, and easy interpretation. Cons include sensitivity to feature scaling and irrelevant features, poor performance in high-dimensional spaces, and large memory requirements to store the dataset. The choice of distance metric heavily influences results, and ties must be resolved deterministically. Noise in the data can disproportionately affect predictions since a single close point dominates.

Variants and efficiency: Weighted 1NN assigns greater influence to closer neighbors, though with k=1 it reduces to the basic rule. Efficiency can be improved with data structures such as k-d trees or ball trees, or through approximate nearest neighbor search. Dataset compression or prototype methods can also reduce storage needs while aiming to preserve accuracy.

Applications and history: 1NN is a foundational baseline in pattern recognition and machine learning. It is the k-nearest neighbors family with k set to one and is often used for quick, non-parametric classification or regression tasks. The method illustrates the broader concept of instance-based learning.