Fleiss

Fleiss is a surname of Germanic origin and may refer to people who bear the name. In statistical literature, the term most often denotes Fleiss' kappa, a measure of inter-rater reliability for nominal data named after Joseph L. Fleiss, who introduced it in 1971. The statistic provides a way to quantify how much agreement among a fixed number of raters exceeds what would be expected by chance.

Fleiss' kappa applies to scenarios with multiple raters (more than two) who classify each item into one of several categories. For each item, the counts of ratings in each category are used to compute the itemwise agreement, and these are averaged across all items to obtain the overall observed agreement. Separately, the overall distribution of ratings across categories is used to estimate the agreement expected by chance. The kappa statistic is then calculated as k = (P̄ − P̄E) / (1 − P̄E), where P̄ is the average observed agreement and P̄E is the expected agreement by chance.

Interpretation guidelines are approximate and context-dependent. Higher values indicate stronger agreement beyond chance, values near zero suggest agreement close to what would be expected by chance, and negative values indicate agreement less than chance. Fleiss' kappa is widely used in psychology, medicine, content analysis, and other fields that require assessment of reliability across several raters.

Limitations include sensitivity to category prevalence, the number of categories, and the distribution of ratings. It assumes independence of ratings and can be affected by marginal imbalances, so results should be interpreted with caution alongside other reliability indicators.