Categorymost

Categorymost is a simple aggregation method used in categorization tasks to assign a single label to each item by selecting the category that appears most frequently among multiple labeled predictions or sources. It is commonly applied in crowdsourcing, ensemble classification, and data annotation workflows where several judgments are available for each item.

Mechanism: For a given item, the labels produced by different annotators or models form a multiset of categories. Categorymost chooses the mode of that multiset. In the event of a tie among two or more categories, practitioners may resolve it by random tie-breaking, a fixed priority order, or by consulting a secondary criterion such as annotator reliability, label confidence, or prior probabilities.

Variants and use cases: Weighted categorymost extends the basic rule by counting labels with weights. Hierarchical categorymost operates within a category hierarchy, selecting the highest-level category that is most frequent. It is used for dataset labeling in machine learning, content tagging, moderation, and quality control in crowdsourcing platforms.

Limitations: Categorymost assumes that all sources are equally reliable and ignores varying confidence levels. It can be biased in the presence of prevalent classes and may fail to reflect nuanced disagreements. Mitigations include weightings, reliability modeling, or combining with soft-voting methods that incorporate probability estimates.

See also: majority vote, ensemble learning, label aggregation, crowd annotation.