| Home > Newsevents > Training > Rcourse_notes > GENERAL_CIRCULATION > GENERAL_CIRCULATION > |
Predicting
uncertainty in forecasts of weather and climate
(Also published as ECMWF Technical Memorandum No. 294)
By T.N. Palmer
Research Department, ECMWF
November 1999
| GENERAL CIRCULATION | |||||
|
Table of contents >>
Next Section >>
1 . Introduction
1.1 Overview
A desirable if not necessary characteristic of any physical model is an ability to make falsifiable predictions. Such predictions are the life blood of meteorology and climate science. Predictions from vast computer models of the atmosphere, integrating the Navier-Stokes equations for a three dimensional multi-constituent multi-phase rotating fluid, and coupled to a representation of the land surface, are continually put to the test through the daily weather forecast (e.g. Bengtsson, 1999; see also http://www.ecmwf.int). On seasonal to interannual timescales, these same models, with 2-way coupling to similar mathematical representations of the global oceans, predict the development of phenomena such as El Niño, with consequences for seasonal rainfall and temperature patterns around much of the globe (e.g. Stockdale et al., 1998; see also http://www.iges.org/ellfb). Coupled ocean-atmosphere models are also widely used to make predictions of possible changes in climate over the next century as a result of anthropogenic influence on the composition of the atmosphere (e.g. IPCC, 1996).
However, there is little sense in making predictions without having some prior sense of the accuracy of those predictions (Tennekes, 1991); quantification of error is a basic tenet in experimental physics. Earth's climate is a prototypical chaotic system (Lorenz, 1993), implying that its evolution is sensitive to the specification of the initial state; however, an appreciation of the importance of quantifying the role that initial error plays in limiting the accuracy of weather predictions pre-dates the development of chaotic models (Thompson, 1957).
How would one go about making an a priori assessment of the accuracy of a weather forecast, or a prediction of El Niño? Of course, a 'climatological-mean' error can be derived by verifying a past set of predictions, and averaging the resulting forecast errors. However, such a crude estimate may not be particularly useful. Chaotic dynamics implies not only that a forecast is sensitive to initial error, but also that the rate of growth of initial error is itself a function of the initial state (see Section 2). Weather forecasters have a practical sense of this dependence of error growth on initial state; certain types of atmospheric flow are known to be rather stable and hence predictable, others to be unstable and unpredictable. As such, a key to predicting forecast uncertainty lies in the estimation of the effects of local instabilities in regions of phase space through which a forecast trajectory is likely to pass.
In addition to error in initial conditions, the accuracy of weather and climate forecasts are influenced by our ability to represent computationally the full equations of that govern climate. For example, there will be inevitable errors in representing circulations on scales comparable with or smaller than a model's truncation scale. These errors can propagate upscale and influence weather and climate phenomena with characteristic size much larger than the truncation scale. Uncertainty in model formulation is certainly one of the most important factors which undermine confidence in climate forecasts - representation of cloud systems (in models which cannot resolve individual clouds) being a particular manifestation of this problem. As with initial error, uncertainties in model formulation impact on climatic circulation patterns through the projection of these uncertainties onto flow-dependent dynamical instabilities of the climate system.
In the body of this paper, results are shown from a number of numerical models of the climate system. It is useful to consider three types of model, distinguished by their degree of complexity. The first could be thought of as 'toy' models; they are used primarily to illustrate particular paradigms. Examples are the Lorenz (1963) model and the delayed oscillator model (see Sections 2 and 7). The second type of model could be described as 'intermediate'; it certainly has prognostic value, but is based on simplified equations of motion where terms which are second order in some small parameter are ignored. For many examples discussed in this paper, the so-called Rossby number
(e.g. Gill, 1982) is
such a parameter. Here, 
, 
denote a typical horizontal velocity and length scale associated with a
particular climatic or weather phenomenon, 
is the Coriolis parameter, where 
is the angular speed of the Earth and 
denotes latitude. Examples of intermediate models, are the atmospheric quasigeostrophic
model (e.g. Marshall and
Molteni, 1993: see Sections
3 and 7), and a simplified
coupled oceanatmosphere model of El Niño (e.g. Zebiak
and Cane, 1987: see Sections
5 and 7). Intermediate
models are generally truncated to have O(103) or less degrees
of freedom, which makes numerical integration and stability analysis extremely
tractable by modern computing standards. The final type of model in the hierarchy of complexity are the comprehensive global climate and weather prediction models; these typically have O(106-107) degrees of freedom. At national (and international) meteorological and climate centres, quantitative weather and climate predictions are now almost universally based on output from these types of model. The models are formulated using finite (Galerkin) truncations of fluid-dynamic partial differential equations where (at most) only the hydrostatic assumption is applied to filter meteorologically-unimportant modes. A possible (and easily visualised) representation is in terms of grid points in physical space; a typical resolution would be about 100 km in the horizontal and 1km in the vertical (somewhat finer for weather prediction models, somewhat coarser for climate prediction models with longer integration times). These equations describe the local evolution of mass, energy, momentum and composition, with suitable source and sink terms. The most important atmospheric composition variable is water, represented in each of its different phases. Details of these equations can be found in many references (e.g. Trenberth, 1992). Such comprehensive models are integrated on supercomputers, with (at the time of writing) typical sustained speeds of O(1011) floating point operations per second. In practice, the difference between the atmosphere component of weather and climate prediction models is not great - and in some instances there is no difference; however, weather prediction models do not generally have an interactive ocean, whilst climate models do. An example of this third type of comprehensive model, discussed below, is the European Centre for Medium-Range Weather Forecasts (ECMWF) weather and climate prediction model (Bengtsson, 1999).
In this paper, we consider two types of prediction. Following Lorenz (1975), we refer to initial value problems as 'predictions of the first kind'. By contrast, forecasts which are not dependent on initial conditions, for example predicting changes in the statistics of climate as a result of some prescribed imposed perturbation, would constitute a 'prediction of the second kind'. A weather forecast is clearly a prediction of the first kind; so is a forecast of El Niño, referred to as a climate prediction of the first kind. By contrast, estimating the effects on climate of a prescribed volcanic emission, prescribed variations in Earth's orbit (thought to cause ice ages) or prescribed anthropogenic changes in atmospheric composition, would constitute a climate prediction of the second kind.
1.2 Scope
This paper deals with the problem of forecasting uncertainty in weather and climate prediction from its theoretical basis, through an outline of practical methodologies, to an analysis of validation techniques including estimates of potential economic value. The author hopes that the mathematical description of these components will be of some help to readers wishing to gain some introduction to the quantitative methods used in the subject. However, at the least, the reader will be able to deduce that the topic of weather and climate prediction is quantitative and objective. (The days are over, of hanging out the seaweed, examining the size of molehills, or studying animal entrails for portents of coming tempests - that is, unless the computers are down!) On the other hand, readers not interested in the details of the mathematics should be able to appreciate many of the results given without dwelling on the equations at any length.
In Section 2, we consider how to forecast uncertainty in a prediction of the first kind, assuming a perfect deterministic forecast model. The evolution equation for the probability density function (pdf) of the climate state vector is the Liouville equation; an example of its solution is given for illustration. However, application to the real climate system is severely hampered by two fundamental problems. The first is directly associated with the dimensionality of the climate equations; as mentioned above, current numerical weather prediction models comprise O(107) individual scalar variables. The second problem (not unrelated to the first) is that, in practice, the initial pdf is not itself well known.
To amplify on this last remark, a description of current (variational) meteorological data assimilation schemes is described in Section 3. These schemes are used to determine initial conditions for weather and climate forecasts, given a set of atmospheric and oceanic observations whose density is heterogeneous in both space and time. Such data assimilation schemes are based on minimising a cost function which combines these observations with a background estimate of the initial state provided by a short-range model forecast from an earlier set of initial conditions. In principle, given Gaussian error statistics, the Hessian or second derivative of the cost function determines the initial pdf. In practice, there are significant shortcomings in our ability to estimate this pdf.
The number of degrees of freedom in comprehensive climate and weather prediction models is not determined by any scientific constraint (there is no obvious 'gap' in the energy spectrum of atmospheric motions), but rather by the degree of complexity than can be accommodated using current computer technology. As such, there are inevitably processes occurring in the atmosphere and oceans which are partially resolved or unresolved and must be represented by some parametrised closure approximation. Examples are associated with cloud formation and dissipation, and momentum transfer to the solid earth by topography. However, there is a fundamental indeterminacy in the formulation of these parametrisations since there is no meaningful scale separation between resolved and unresolved scales in the climate system. Section 4 describes two recent attempts to represent the pdf associated with this uncertainty in the computational representation of the equations of motion of climate: the multi-model ensemble, and stochastic parametrisation.
A theoretical framework for describing error growth is developed in Section 5. Two common measures of perturbation amplification used in different branches of physics and mathematics are normal mode growth and Lyapunov exponent growth. Neither is well suited to describing error growth in the climate system. Firstly, because of the advective nonlinearity in the governing equations of motion, the linearised dynamical operators are not normal; as such, over finite times, perturbation growth need not be bounded by the fastest eigenmode growth. Also, dominant Lyapunov or eigenmode growth in a comprehensive multi-scale model may refer to fast instabilities (such as convective instabilities) whose spatial scales are much smaller than those describing weather or climate phenomena. To address these problems, we discuss in Section 5 a general formulation of perturbation growth in the linearised approximation, in terms of a singular value decomposition of the linearised dynamics (building on the developments in Section 3). Examples of singular vectors for weather and climate prediction problems are shown, and their fundamental non-modality is discussed. Because of the nonlinearity of the underlying dynamics, the appropriate singular values vary on the attractor; this variation describes why forecast error can fluctuate for fixed initial error. The variation of singular values on the attractor is also relevant for understanding the amplification of model error by flow dependent instabilities. The relationship between singular values, eigevalues and Lyapunov exponents is discussed.
Section 6 discusses some applications of the singular vector analysis. In one application ('chaotic control of the observing system') singular vectors are used to determine locations where additional 'targeted' observations might significantly improve a forecast's initial state.
Section 7 describes the basis behind attempts to predict uncertainty in daily, seasonal and climate change forecasts using ensembles of atmosphere or coupled ocean-atmosphere model integrations. In practice such ensembles are interpreted in probabilistic form. If the ensemble of forecast phase-space trajectories evolve though a relatively stable part of the climate attractor, then resulting probability forecasts will be relatively sharp. Conversely, if the ensemble passes through a particularly unstable part of the attractor, then the corresponding forecast probability may be little different from a long-term climatological frequency.
The question of how to validate probability forecasts is discussed in Section 8. Two particular techniques are described. The first is based on a root mean square distance between the probability forecast of a dichotomous event and the corresponding verification. This measure allows one to formulate the notion of reliability of probability forecasts. The second quantity measures the so-called hit and false alarm rate of the forecast of a dichotomous event, assuming that the event is forecast if the predicted probability exceeds some prescribed probability threshold.
A fundamental question when assessing probability forecasts is whether a useful level of skill has been attained. Obviously, different users have different criteria for judging usefulness. For some, probability forecasts might be deemed useless unless they are sharp and quasi-deterministic. For others, who might be looking to accrue benefit over a long time, forecast probabilities which are only marginally different from climatological frequencies, may be useful. To assess this issue more quantitatively, a simple cost/loss decision model is applied in Section 9 based on the hit and false alarm rates discussed in Section 8. It is shown, that the (potential) economic value of probability weather forecasts for a variety of users, is higher than the corresponding value from single, deterministic forecasts.
Concluding remarks are made in Section 10.
Training Course Notes Front Page >>
Table of contents >>
Next Section >>
Copyright © 2003, ECMWF. All rights reserved.
 |
13.05.2003 |
 |
© ECMWF |
