relabels

Relabeling, sometimes described by the present tense verb form relabels, refers to the process of updating the labels assigned to data instances in a dataset. It is used to correct labeling errors, refine a label taxonomy, or adapt to new guidelines and domain knowledge. Relabeling can affect any data modality—images, text, audio, or structured records—and is common in both research datasets and production pipelines. The result is a revised label assignment, potentially altering class distributions and downstream model training.

The workflow typically combines human oversight with automated checks. It begins with updated annotation guidelines or a new codebook, followed by a review phase that monitors inter-annotator agreement. Techniques include manual relabeling by domain experts, crowd-sourced relabeling, and model-assisted relabeling that flags uncertain items for human review. Data provenance and versioning are important to document which items were relabeled and why. Active learning and weak supervision can prioritize samples most likely to improve model performance when relabeled.

Impact and evaluation: Relabeling can improve accuracy, fairness, and robustness but also risks inconsistency and label drift if not managed carefully. Evaluation uses agreement metrics such as Cohen's kappa or Krippendorff's alpha and downstream validation to assess effect on tasks. Best practices include documenting changes, preserving original labels for audit, and testing on representative hold-out data. Align relabeling with domain standards and stakeholder goals, and maintain transparent records to preserve reproducibility.