wellcalibrated

Well-calibrated is an adjective used to describe a probabilistic system, model, or measurement whose outputs align with observed frequencies or true values. In statistics and machine learning, a model is well-calibrated if, among all instances given a predicted probability p, approximately p proportion actually belong to the positive class. Calibration is about aligning beliefs with reality, not merely maximizing accuracy.

In measurement and instrumentation, well-calibrated instruments produce readings that agree with traceable reference standards across the operating range. Calibration often involves adjusting the device and documenting the process to ensure traceability and minimize bias.

In machine learning, common calibration techniques include Platt scaling, isotonic regression, and temperature scaling, sometimes applied post hoc to adjust predicted probabilities. Calibration can also be improved during training with proper loss functions or calibration-aware objectives.

Calibration is assessed with reliability diagrams, calibration curves, Brier score, and other metrics. It is distinct from discrimination, the ability to separate classes, and from overall accuracy; a model can be well-calibrated yet have poor discrimination, or vice versa.

Calibrated probabilities are important for decision making under uncertainty, risk assessment, and decision thresholds. Calibration can degrade under distribution shift or with small sample sizes, requiring ongoing monitoring and re-calibration.