S3VM

S3VM, short for semi-supervised support vector machine, is a family of machine learning algorithms that extends the standard support vector machine to leverage both labeled and unlabeled data during training. The aim is to improve classification performance when labeled data are scarce by exploiting the structure of the unlabeled examples.

In a typical S3VM setup, the training set consists of labeled instances (xi, yi) with yi in {−1, +1} and unlabeled instances xj. The learning objective augments the ordinary SVM margin term with a contribution from the unlabeled data, which introduces latent labels yj ∈ {−1, +1} for the unlabeled points. A common formulation minimizes the margin risk plus regularization, subject to constraints that labeled points satisfy the margin and unlabeled points are assigned one of the two classes. Since this requires choosing the latent labels, the problem is inherently non-convex and is usually solved via alternating optimization: initialize the unlabeled labels (often by a preliminary classifier), train an SVM, reassign the unlabeled labels based on the current decision function, and iterate. Some approaches replace hard latent labels with soft or probabilistic assignments to improve tractability.

S3VM is related to, and sometimes compared with, transductive SVM (TSVM) and other semi-supervised methods that propagate information from labeled to unlabeled data. It is particularly useful when the data distribution allows unlabeled examples to inform the decision boundary, such as when the decision boundary should avoid dense regions of the input space.

Limitations include non-convex optimization, sensitivity to the unlabeled data distribution, and potential degradation if unlabeled data are not representative. Computational cost is also higher than standard SVM due to the latent-label optimization.