Use and Verification of ECMWF Products in Member and Co-operating States (2018)

Each summer ECMWF invites Member and Co‐operating States (i.e. National Meteorological Services (NMS)) to submit updated reports on the application and verification of ECMWF’s forecast products. This report summarises the findings from the 2018 reports, in conjunction with additional feedback gathered in other fora.
Direct use of ECMWF products remains extensive within the Member and Co‐Operating States, spanning time ranges from day‐1 to seasonal. The peak in direct usage is for lead times of about three to five days, when initial warnings for potential high impact events are commonly considered. At short ranges, up to about day‐3, ECMWF model data commonly also provides boundary conditions for deterministic and ensemble limited area model (LAM) runs. Consortia are increasingly being developed across Europe, to perform LAM runs centrally for groups of countries. For example, the Nordic MetCoOp project, which runs the AROME 2.5 km model in ensemble mode, is expected to have expanded from the current three partner countries to eight countries by 2022.
In the last couple of years, the average resolution difference between operational LAMs and everyday output from ECMWF’s Integrated Forecast System (IFS) has increased again; 2 km LAM runs are now typical. Probably because of this difference, new verification results show slightly higher skill levels, overall, for LAMs compared to HRES, for surface weather parameters. However, this statement hides what is a very complex picture, with relative performance depending strongly on the LAM itself, on region, on season, on time of day, on weather type and on weather parameter. Two key issues with ECMWF model output continue to be wind strengths over mountains and screen temperatures in very stable conditions. These are being actively worked on at ECMWF. Many NMSs continue to use statistical post‐processing to successfully address these and other bias issues.
Most NMSs indicate that they are extremely satisfied with ECMWF operational output. In conjunction, expectations have grown, to the extent that in some fields they can exceed what is physically reasonable. Our inability to pinpoint windstorms, to predict very low visibility in fog, to forecast waves near complex coastlines and to deliver accurate seasonal forecasts have all been noted. ECMWF probably needs to better manage expectations in very challenging areas such as these.
Recent ECMWF product initiatives have been widely praised; e.g. vertical profiles, lightning diagnostics and, most notably, precipitation type meteograms. The ecCharts tool is very widely used, although speed continues to be an issue for some.
ECMWF will continue to deliver efficiency savings for the Member and Co‐operating States, by serving as many user requests as possible in the coming months and years, subject to the decisions of ECMWF governing bodies, and to available computational and human resources. At the same time issues with model output will be investigated and addressed as far as possible. ECMWF appreciates and strongly encourage the continued feedback of issues identified by forecasters and others. This mechanism continues to be very effective in initiating an investigative process that eventually improves forecast quality.