pseudolabels

Pseudolabels are labels assigned to unlabeled data by a model’s own predictions, used as targets in training. In semi-supervised learning, they enable models to learn from large amounts of unlabeled data by augmenting the labeled dataset with these self-generated labels. The typical workflow involves training a model on a small labeled set, using the trained model to infer labels for unlabeled examples, selecting examples with high confidence as pseudolabeled data, and retraining the model on the combined dataset. This process can be repeated iteratively. In some setups, a teacher model generates pseudolabels for a student model, forming a teacher–student framework.

Quality and risks: Pseudolabels rely on the model’s predictions, so incorrect pseudolabels can reinforce errors (confirmation bias). Confidence thresholds, temperature calibration, data augmentation, and ensemble methods are used to improve reliability. Pseudolabels can also introduce bias if the labeled and unlabeled data are drawn from different distributions. Class imbalance and label noise are common concerns, often addressed by weighting, curriculum learning, or using only the most confident predictions.

Applications: Pseudolabeling is widely used in computer vision and natural language processing, as well as speech and time-series tasks, where unlabeled data is abundant. It is often combined with other semi-supervised techniques, such as consistency regularization, to promote stable predictions under perturbations.

Limitations: Not a substitute for labeled data in all scenarios; effectiveness depends on data similarity, label noise tolerance, and model calibration. In practice, pseudolabeling is one component of a broader semi-supervised learning strategy.