newsletter-banner-NL155

ECMWF releases Atlas software library

Willem Deconinck

 

ECMWF has publicly released Atlas, a software library supporting the development of Earth system model components and data processing, with an open source Apache 2 licence. The source code can be downloaded freely at www.github.com/ecmwf/atlas and contains installation instructions.

Emerging diverse and complex hardware solutions have a large impact on the programming models traditionally used in software for numerical weather prediction (NWP). Furthermore, NWP models are becoming increasingly complex as more Earth system components are introduced. Each model component may use different numerical algorithms and may be based on structured or unstructured grids.

In response, ECMWF is developing Atlas, an object-oriented (OO) programming library for developing flexible next-generation NWP models on existing and emerging hardware, such as GPUs. Atlas addresses the complexities associated with managing parallel distributed data structures on Earth grids. It provides mesh-generation capabilities from a wide catalogue of grids. Through its OO design, the memory layout and parallelisation of fields defined on meshes can be abstracted to accommodate specific numerical methods and hardware implementations. Atlas not only supports structured global grids such as the ones used operationally at ECMWF, but also unstructured meshes and structured regional grids as used in limited-area models. Although Atlas’s main features are implemented using C++, a modern OO Fortran interface allows Atlas features to be used in most existing NWP software packages.

%3Cstrong%3E%20Distributed%20mesh%20generation.%20%3C/strong%3E%20Atlas%20is%20here%20used%20to%20generate%20a%20distributed%20mesh%20from%20a%20very%20coarse%20reduced%20octahedral%20Gaussian%20grid.%20Each%20distributed%20mesh%20partition%20can%20be%20constructed%20with%20parallel%20halos%20to%20allow%20stencil%20operations.%20Distributed%20fields%20can%20then%20be%20created%20based%20on%20a%20specific%20%E2%80%98function%20space%E2%80%99,%20which%20is%20tightly%20coupled%20to%20numerical%20schemes%20of%20how%20discrete%20data%20is%20interpreted%20in%20a%20continuous%20space.
Distributed mesh generation. Atlas is here used to generate a distributed mesh from a very coarse reduced octahedral Gaussian grid. Each distributed mesh partition can be constructed with parallel halos to allow stencil operations. Distributed fields can then be created based on a specific ‘function space’, which is tightly coupled to numerical schemes of how discrete data is interpreted in a continuous space.

At ECMWF, Atlas is being used in the development of a finite-volume module (FVM) for the Integrated Forecasting System (IFS) (see ECMWF Newsletter No. 145, pp. 24–29). Its parallel mesh-generation capabilities build a distributed unstructured mesh about the grid points of the IFS’s reduced Gaussian grid. The resulting triangular or quadrilateral elements are then used to evaluate finite-volume differential operators. Atlas’s mesh generator is also used in ECMWF’s new Meteorological Interpolation and Regridding (MIR) software (see ECMWF Newsletter No. 152, pp. 36–39), where the resulting elements are used to construct linear interpolation operators.

Atlas will be crucial in longer-term IFS developments. For example, in the EU-funded Horizon 2020 ESCAPE project it serves as a collaborative framework to develop next-generation building blocks, dubbed NWP and climate ‘dwarfs’, for NWP and climate models. In ESCAPE, these dwarfs are studied in isolation, optimised and ported to alternative hardware solutions, such as Intel’s Xeon Phi and NVIDIA’s GPU. Atlas’s data structures and parallelisation routines therefore support synchronisation between CPU and GPU memory spaces.