More than 100 online and 47 in-person participants attended an ECMWF event on model uncertainty in weather forecasting from 9 to 12 May 2022.
They listened to 22 talks by experts from seven countries, who gave a wide-ranging view of the state of the art. They also attended poster sessions and discussion groups to see how work on model uncertainty can be taken further.
On opening the workshop, ECMWF Director of Research Andy Brown stressed the importance of the topic for ensemble forecasting, in which several slightly different forecasts are run to present the range of future weather possibilities.
“We have a real commitment to ensembles and to probabilistic forecasting, and as a result to model uncertainty representation,” he said. “It is particularly timely to have this workshop as there are a lot of open questions.”
Among them he highlighted the representation of model uncertainty in different parts of the Earth system, such as the land, the ocean, and sea-ice as well as the atmosphere, and in convection-permitting global Earth system models.
ECMWF Director of Research Andy Brown opened the four-day workshop.
Main topics
At the start of the workshop, ECMWF scientist Martin Leutbecher proposed a set of six guiding principles for developing model uncertainty (MU) representations. These cover their design (state-dependence and physical consistency), their application (controllability, numerical stability and computational efficiency) and their performance (forecast skill enhancement).
Tim Palmer (University of Oxford) reminded the audience that a major source of model uncertainty in atmospheric forecast models arises from the need to describe the sub-grid atmosphere by simplified processes (parametrization). The resulting truncation-related errors led to the development of the Stochastically Perturbed Parametrization Tendencies scheme (SPPT). SPPT was first implemented in ECMWF’s Integrated Forecasting System (IFS) ensemble in 1998. It has brought significant probabilistic forecast skill ever since.
Several groups showed recent results with SPPT. Developments at the US National Oceanic and Atmospheric Administration (NOAA) are addressing moisture and energy conservation problems when using SPPT for the atmosphere. Two groups – one using NOAA’s GFS model and one using NEMO – shared early results from extending SPPT to describe MU in the oceans.
Poster sessions provided ample opportunities for an informal exchange of ideas.
New and alternative descriptions of MU for the atmosphere featured widely, including several presentations demonstrating the growing use of the Stochastically Perturbed Parametrizations (SPP) scheme. Promising results were presented for the IFS, HarmonEPS, the Canadian GEPS, and FV3-LAM models, among others.
With an eye on the future for high-resolution forecasting at ECMWF, several presentations demonstrated the challenges for limited-area models operating at convection-permitting scales. The re-balancing of contributions from atmospheric components when the convection scheme is switched off and the dominant effects of initial and lateral boundary conditions require a reassessment of MU representation.
Several talks highlighted progress with stochastic sub-grid parametrization schemes. A stochastic shallow convection scheme is scheduled for inclusion in the ICON model at DWD, and a method using cellular automata to simulate the evolution of sub-grid clouds will soon be used in the GFS at NOAA. In addition, two new approaches for including stochastic boundary layer perturbations were shared, with results from testing the PSP2 scheme in ICON and the SBL scheme in MOGREPS-UK.
MU descriptions for other components of Earth system models were discussed. These include working from a stochastic description of the atmospheric governing equations, and, for the coupled components of the Earth system, ocean stochastic physics, stochastic sea-ice and ocean parametrizations, and accounting for MU in the land surface representation for coupled (land–atmosphere) data assimilation and forecasting.
Seminar topics were discussed in working groups, with in-person and remote participation.
Significant discussion focused on the choice of parameters used in MU schemes. These include in particular spatial and temporal correlation scales and random number distributions. Several presentations demonstrated large model sensitivities to the choices made.
The new ‘MU-MIP’ project was highlighted. It is designed to help constrain these free parameters through analyses of model errors, as estimated from tendency differences using a high-resolution simulation as a substitute for the real atmosphere.
There was universal agreement that we must collectively place greater efforts on the evaluation of MU schemes: to develop objective measures for identifying relevant perturbations; to understand the characteristics of the perturbations; to assess their impact across forecast lead-times, regions and weather regimes; and to enable meaningful comparisons of their performance across models.
The presentations featured several approaches, including metrics for assessing spatial ensemble spread in km-scale ensembles, and using observations to validate stochastic perturbations via ensemble covariances.
Outlook
“This was a very successful workshop due to the breadth and quality of the talks and posters and the contributions, both in-person and online, to the workshop discussions,” said ECMWF scientist Sarah-Jane Lock.
The outcomes of the event will feed into model development at ECMWF. “We are working on improving the representation of model uncertainty in the IFS, including through replacing our current Stochastically Perturbed Parametrization Tendencies (SPPT) scheme by the Stochastically Perturbed Parametrizations (SPP) scheme. The talks and discussions at this event have given us an excellent basis for further developments.”
A full set of talk recordings and presentations is provided on the event website. Reports from the three working group discussions will also be published on the event website in due course.
Group photo of in-person participants at the Model Uncertainty Workshop from 9 to 12 May 2022.