Classifier

A classifier is a model used in statistics and machine learning to assign discrete class labels to inputs. Given a feature vector, the classifier outputs a predicted label. Training involves fitting the model to a labeled dataset, learning the relationship between features and the target class. Unlike regression, which predicts continuous values, classification predicts categories.

Classification methods can be supervised and probabilistic or discriminative. They can be binary (two classes) or multiclass. Common algorithms include logistic regression, Naive Bayes, decision trees, random forests, support vector machines, k-nearest neighbors, and neural networks. Some methods, like linear discriminant analysis and quadratic discriminant analysis, model the distribution of features per class. Many classifiers can produce probability estimates for class membership.

The training process typically uses a training set to fit model parameters and a validation set to tune hyperparameters. Performance is evaluated with metrics such as accuracy, precision, recall, F1 score, and area under the ROC curve. Calibration may be used to adjust predicted probabilities to reflect true frequencies.

Applications span email spam filtering, image and speech recognition, medical diagnosis, fraud detection, and document tagging. Challenges include limited or biased data, class imbalance, overfitting, feature selection, and interpretability of the decision rule.

Related concepts include decision boundaries in feature space and the idea that some classifiers are probabilistic while others provide hard class labels. In some contexts, a classifier is part of a larger pipeline that includes data preprocessing, feature extraction, and post-processing.