New IFS version control and issue tracking tools

As part of continuous efforts to improve our software infrastructure, and therefore research productivity, we have migrated the Integrated Forecasting System (IFS) code from the Perforce version control system (VCS) to Git. We have also introduced the JIRA tool for issue tracking and software management.

Git

The migration to Git was started during the development of IFS Cycle 43r3, which was implemented on 11 July 2017 (see separate article in this Newsletter). In a phased approach, researchers developed their individual contributions under Perforce, but then the branches were imported into Git, which was used to produce the final build. IFS-specific extensions to Git were developed and tested during this period. The next operational cycle, 45r1, will be the first to be developed end-to-end in Git.

Git has several advantages for ECMWF and Member State developers of the IFS. As it is a distributed rather than centralised VCS, each developer holds their own copy of the full repository, complete with its entire history. Consequently most version control operations are local and do not involve communication with a central server. Hence:

• Performance will not degrade with the size of the repository or number of users
• Committing changes, creating branches and merging are much faster, which will encourage better software development habits
• Git can be used without network access – a connection to the server is needed only to share work with others, which happens less often
• Developers can work locally on multiple machines, e.g. when hot-desking or travelling
• There is more control over sharing, leading to a cleaner organisation of branches, rather than a single repository with tens of thousands of branches exposed to every user.

Moreover, Git is free software, thus opening the possibility of cost savings. It is also widely known, thus facilitating sharing code with other organisations and accepting external contributions.

JIRA

Starting with Cycle 43r3, we have also taken the opportunity to introduce JIRA, a web-based issue tracking and software project management application. JIRA is part of the same suite as Bitbucket, which ECMWF already uses to host all its software (including now the IFS, as a result of the Git migration), and Confluence, which ECMWF uses as a wiki and to host its intranet.

This now provides a central repository of information for IFS development. It documents modifications, from bug fixes to major scientific improvements, in a systematic way, providing a widely available, transparent, searchable database. Each code change is logged with information about the change, its interactions with other developments, the testing and evaluation that has been performed, and other details such as dependencies on other software components. This leads to much more transparency and clarity about what work is being done, by whom, and what state it is in. ECMWF already hosts its User Request Management System in JIRA, so now an IFS code change for a new output parameter, for example, can be linked within JIRA to the Member State request, giving a full line of sight. It can also be linked to Data Governance requests, to ensure these inter-dependencies are managed.

This new system will facilitate project management and delivery of cycles; improve communication within and between ECMWF departments, Member and Co-operating States, and external collaborators; and centralise and streamline IFS-related documentation.