OvR

OvR, short for one-vs-rest, is a strategy for multiclass classification that reduces a multiclass problem to multiple binary classification tasks. For a dataset with k classes, OvR trains k binary classifiers, each dedicated to distinguishing one target class from all other classes combined.

In training, each binary classifier is built to separate its designated class from the rest. In prediction, all k classifiers are applied to a new instance. Each classifier outputs a score or probability indicating the likelihood that the instance belongs to its class, and the final predicted class is typically the one with the highest score or probability. When probabilities are used, calibrating them across classifiers can improve comparability.

Common base classifiers used in OvR include logistic regression, linear support vector machines, and decision trees. The approach is convenient because it can reuse well-understood binary models and generally scales reasonably with the number of classes. The overall training cost is roughly k times the cost of a single binary model.

Relation to OvO: One-vs-rest contrasts with one-vs-one (OvO), where a classifier is trained for every pair of classes. OvO leads to k(k−1)/2 binary models, which can be more accurate in some settings but more computationally intensive to train and predict with as k grows.

Implementation notes: In practice, OvR can be implemented by wrapper methods such as OneVsRestClassifier in machine learning libraries. Alternatives include OvO classifiers like OneVsOneClassifier. OvR is commonly used with linear models and is a standard baseline approach for multiclass problems.