Rank histogram (Talagrand diagram)

A more detailed way of validating the EPS spread is by a rank his­togram (sometimes called a Talagrand diagram). It is constructed from the notion that in an ideal EPS system the verifying analysis is equally likely to lie in any “bin” defined by any two ordered adjacent members, including when the analysis is outside the ensemble range on either side of the distribution. This can be understood from induction, if we consider an ideal EPS with one, two or three members:

With one ensemble member ( I ) verifying observations (●) will always (100%) fall “outside” ●I●

With two ensemble members (I I), verifying observations will for this ideal EPS fall outside in two cases out of three  ●I●I●

With three ensemble members (I I I), verifying observations will fall for this ideal EPS outside in two cases out of four ●I●I●I●

In general, if N = number of members, the verification will in two cases out of N + 1 always fall outside, yielding a proportion of 2/(N+1) outside. For the same reasons the operational deterministic forecast and the EPS Control should lie outside the ensemble 2/(N+1) of the time. For a 50-member ensemble system this means 4%. This is consistent with the discussion in Section 4.4.8, that due to the limited number of ensemble members, it would be unrealistic to assume that the probability was 0% or 100% just because none or all of the members forecast the event.

In an ideal EPS system, the rank histogram distribution should, on average, be flat with equal numbers of verifying observations in each interval. If there is a lack of spread, this will result in a U-shaped distribution with an over-representation of cases where the verifications fall out­side the ensemble and under-representation of cases when they fall within the ensemble centre. If the system has a bias with respect to the verifying parameter, the U-shape might degenerate into a J-shape.

An ideal ensemble system might, however, display a U-shape distribution due to observation uncertainties. For example, with 50 ensemble members an ensemble spread of 20°C yields an average bin width of  0.4°C, an ensemble spread of 5°C yields an average bin width of only 0.1°C, smaller than the observation uncertainty (see Figure 86).

Figure 86:  An observation (a filled circle) and its uncertainty assumed symmetric (the arrows). Since the forecast bins widen their intervals away from the centre (the mean of the distribution), an observation is more likely, for random reasons, to fall into an outer and wider bin than an inner and narrower one.

The small bin size  introduces an element of chance with respect to which bin the observation will fall into. Since the bin sizes due to the normal distribution will increase with increasing distance from the centre, an observation is more likely to end up in a bin further away from the centre than closer to the centre. This will result in a misleading U-shaped distribution.