The development and evaluation process followed at ECMWF to upgrade the Integrated Forecasting System (IFS)

This paper discusses the development and evaluation process followed at ECMWF to upgrade the Integrated Forecasting System (IFS), and illustrates how potential changes, developed and tested by individual scientists, are gradually merged and evaluated prior to their acceptance into the next version of the ECMWF IFS.

It discusses why, and how we are using a hierarchical testing strategy, whereby we test and merge changes stepwise and gradually. Individual changes are tested first in reduced configurations, then if accepted they are merged, and tested in more complete configurations. Finally, all the proposed changes that are deemed acceptable are merged and tested in all the IFS components at operational resolution. Evaluation relies on a use a range of metrics, selected to provide us with meaningful, statistical significant information on the impact of the cycle upgrade.

At the end of the process, a new cycle is judged to be ready for implementation if (a) it brings improvements to the ECMWF forecasts, and/or (b) it improves the realism of the simulated Earth-system, and/or (c) if it includes new forecast products that could improve our service to our users. All three aspects are considered when a final decision is taken. Thus, some model cycles could be implemented even if the impact on forecast quality is small or neutral, provided that new features that could lead to future improvements are implemented.

A focus of the current paper - indeed the primary driver for it - is to review whether the details of the hierarchical testing approach are optimal. Some changes are proposed in response to evolving strategic drivers and the increasing complexity of the full system.