interobserver

Interobserver refers to the relationship between two or more observers who make measurements or judgments about the same phenomenon. Interobserver reliability, also called interrater reliability, is the extent to which different observers provide consistent assessments. It is a key indicator of data quality in studies involving subjective judgment, coding schemes, or diagnostic classifications. High interobserver reliability implies that results are reproducible across independent observers, while low reliability indicates measurement error or ambiguous criteria. Intraobserver reliability measures the consistency of a single observer over time.

Measurement of interobserver reliability relies on statistics that quantify agreement. Cohen's kappa is widely used for two raters with nominal data; weighted kappa extends this to ordinal data; Fleiss' kappa handles multiple raters. For continuous measurements, the intraclass correlation coefficient (ICC) is commonly applied. Other measures include Scott's pi and various forms of ICC depending on the model and type of data. Confidence intervals and sample size influence the precision and interpretation of these statistics.

Improving interobserver reliability involves several practices: developing clear, operational definitions and standardized protocols; conducting training and calibration sessions; performing pilot scoring and consensus discussions; blinding raters to each other's assessments when feasible; and using reference standards or gold standards to guide judgments. Ongoing monitoring can help detect and correct rater drift over time.

Applications and limitations: interobserver reliability is essential in medicine (radiology, pathology), psychology, education, content analysis, and field surveys. Limitations include dependence on category prevalence, potential rater bias, and drift in judgments over time. Kappa statistics can be sensitive to base rates, so reporting raw agreement alongside reliability coefficients is common for a complete view.