Miscalibration

Miscalibration refers to the condition in which the output of an instrument, sensor, or model does not correspond to the true value or intended probability. It occurs when a device or system that should produce accurate measurements, probabilities, or decisions produces results that systematically deviate from reality. Calibration is the process of adjusting a measurement or estimation method to align with reference values; miscalibration is the absence or failure of that alignment.

Miscalibration can affect physical instruments, statistical models, risk assessments, and artificial intelligence systems. In physical devices, it may arise from aging, wear, environmental changes, or improper setup. In models, it can result from misspecified structure, training data that do not match deployment conditions, or distribution shift, causing predicted probabilities to be too high or too low relative to observed frequencies.

Detection relies on calibration tests that compare reported values to known references, or reliability diagrams and Brier scores for probabilistic forecasts. Remedies include recalibration, sensor recalibration, updating model parameters, gathering new data, or using calibration techniques such as isotonic regression, Platt scaling, or temperature scaling in machine learning; more generally, adjusting the instrument, applying bias correction, or replacing faulty components.

Miscalibration can lead to systematic errors, overconfidence or underconfidence, and poor decision making across domains such as engineering, medicine, finance, and scientific measurement. Noting and addressing miscalibration is essential for accuracy, safety, and trust in quantitative assessments.