ECMWF has extended the quality monitoring of observations used by the global forecasting system of the Copernicus Atmosphere Monitoring Service (CAMS) by adding an automatic alert system. CAMS is implemented by ECMWF on behalf of the European Commission.
As part of its portfolio, CAMS provides 5-day forecasts of global atmospheric composition. This makes it possible to predict air quality, greenhouse gas concentrations, and the evolution of the ozone hole.
The CAMS global system uses ECMWF’s Integrated Forecasting System (IFS). It employs a wide range of quality-controlled satellite observations to produce the best possible initial conditions (analysis) on which the forecasts are based. The quality of the analyses and the subsequent forecasts depend, amongst other factors, on the quality and availability of observations.
ECMWF has recently included atmospheric composition observations in the automatic observation monitoring system it operates for numerical weather prediction (NWP). This will help to provide a continuous assessment of observation data quality and availability, and to detect any issues in a timely manner.
Standard CAMS monitoring system
The standard observation monitoring system for CAMS global forecasts already provides detailed statistical information on observation values and differences between observations and short-range forecasts.
These differences are called first-guess departures and show up problems in the observations, or in the model, more clearly than absolute values. They are also very useful to assess the quality of new observations.
The statistics of observations and departures are presented in various forms to allow geographical, vertical, and temporal assessment of the data. They are primarily produced to help improve the usage of observations within the CAMS global data assimilation system, but they can also provide useful feedback to data providers about the quality of the observations.
CAMS global monitoring statistics are published on the CAMS website and are accessible to the public. The content is updated routinely every day.
It should be noted that changes in the monitoring statistics are mostly driven by the quality of observations. They may, however, also result from factors not directly related to the observations themselves, such as the model’s inability to represent certain extreme or rare conditions.
CAMS model upgrades or satellite retrieval upgrades will also show up in the monitoring statistics.
This is an example of plots traditionally provided by the CAMS global observation monitoring system. It shows the standard deviation of first-guess departures affecting ozone values obtained from the GOME-2 instrument on the Metop-B satellite from 8 to 11 June 2021. The red patches show clear signs of an increase in the noise from the instrument.
New automatic data checking
With the increasing size and diversity of the global observing system for atmospheric composition, it is difficult to rely solely on manual checking of monitoring statistics.
For many years, an automatic framework has been used at ECMWF to monitor NWP observations used by the atmospheric, land surface and ocean data assimilation systems.
This framework has recently been extended to cover observations used by the CAMS global data assimilation system. The geophysical quantities supported are ozone, carbon monoxide, aerosols, nitrogen dioxide and sulphur dioxide (including volcanic emissions).
Observations of these quantities are provided by several satellite instruments. The automatic checking system is detailed enough to target individual satellite instruments, individual atmospheric layers, and pre-defined geographical areas. The system is designed to easily add support for new sources of observations.
Selected statistical parameters (number of observations, bias correction, mean bias-corrected background, first-guess departures, etc.) are checked against an expected range. An appropriate alert message (including a time series plot) is generated if statistics are outside the specified ranges. A severity level is assigned to each message depending on how much the statistics deviate from the expected values.
Two types of checks are implemented:
- Checks against limits computed dynamically after every analysis cycle using the statistics from the past 20 days and excluding the last two assimilation cycles. These checks are designed to detect sudden changes affecting the statistics.
- Checks against fixed limits computed on demand using long time series. The aim of this test is to detect a slow drift of the statistics.
Warnings are distributed to ECMWF staff to allow follow-up investigations. If necessary, staff can trigger mitigating actions, such as blocking the data from being assimilated over the following days.
The warnings are also expected to identify cases where certain atmospheric composition events are not well captured by the CAMS global forecasts.
Time series of the standard deviation of first-guess departure for ozone obtained from the GOME-2 instrument on the Metop-B satellite. The quality of the data suddenly degraded on 10 June 2021 (see the previous chart). This kind of issue is now expected to be detected by the automatic data checking system.
ECMWF plans for its data checking system to make use of machine learning techniques in the future. The goal is to improve the reliability of the system in detecting issues and, more importantly, to categorise each event appropriately in terms of its severity and likely cause.