BrierScores

The Brier score is a measure of the accuracy of probabilistic forecasts for binary events. For a sequence of N forecasts, each forecast assigns a probability f_i to whether the event will occur, and the observed outcome o_i is 1 if the event occurs and 0 otherwise. The Brier score is defined as BS = (1/N) sum_{i=1}^N (f_i - o_i)^2. The score ranges from 0 to 1, with lower values indicating better predictive accuracy and 0 representing a perfect forecast.

As a proper scoring rule, the Brier score incentivizes honest probability forecasts. If the true probability of the event is q, the expected value of (f - o)^2 is minimized when the forecast probability f equals q. This property makes the Brier score suitable for evaluating probabilistic forecasts rather than point predictions.

The Brier score can be interpreted through a decomposition into reliability (calibration), resolution, and uncertainty. Reliability measures how closely forecast probabilities align with observed frequencies within groups. Resolution reflects the ability of the forecasts to differentiate between events and non-events. Uncertainty depends on the inherent variability of the event outcomes given the base rate. Together, these components help diagnose whether miscalibration or lack of discrimination drives a high score.

Applications of the Brier score are common in weather forecasting and other domains that produce probabilistic binary forecasts, such as rainfall occurrence or system failures. It is often used with a Brier Skill Score to compare forecasts against a reference, such as climatology. Extensions exist for multi-class forecasting through the quadratic score, but for binary events the Brier score remains the standard measure of probabilistic accuracy.