Calibration of probabilities

For operational purposes the reliability can be improved by calibration using verification statistics. For instance, if it is found that in cases when 0% has been forecast, the event tends to occur in 30% of the cases, and when 100% has been forecast, only in 70% of the cases. If the misfit is linearly distributed in between these two extremes, the reliability can be made perfect by calibration - but at the expense of reduced sharpness, since very low and very high robabilities are never forecast (see Figure 92).

Figure 92:  The probability distribution from an overconfident forecast system is calibrated, limiting the range from 0% - 100% to 30% - 70%.

On the other hand, when the probability forecasts are under-confident, calibration might restore the reliability without giving up the sharpness (see Figure 93).

Figure 93:  The probabilities from an under-confident forecast system are calibrated, widening the range from 30% to 70% to 0% to 100%.

This means that there is some “hidden skill” in probability forecasts biased in this way.