Table of contents
Chapter 1. Non-IFS observation processing (OBSPROC): General overview
Chapter 2. Observations:
Types, variables and error statistics
Chapter 3. CMA creation (MAKEMA)
Chapter 4. The FEEBACK task
Chapter 5. The TOOLS task
Chapter 6. Central-memory
array (CMA) structure/format
Chapter 7. BUFR feedback data structure/format
Chapter 8. SIMULATED-observations data structure/format
Chapter 9. NAMELISTS
Chapter 10. Processing of scatterometer data
REFERENCES
|
|
Next
Section
Previous Section
As explained in Section 1.5,
after finding out in the main controlling subroutine CNTOBSPR that the MAKECMA
task is to be run, and after branching off into that section of the subroutine,
the real MAKECMA task can start (see
Fig. 1.3 ). If the MAKECMA task is to be run in parallel mode, subroutine
PREPROC_MPP_MAKECMA is called next. It is this subroutine which will by
a number of the OBSORT subroutine calls invoke the parallel processing.
Somewhere down in the OBSORT the input BUFR files are then read in and BUFR
messages distributed across PEs (to get an initial load balance) and eventually
written out as one `sorted BUFR file' per PE. Once the PREPROC_MPP_MAKECMA
has finished, the control is returned back to the CNTOBSPR. The next step
is to call MAKECMA subroutine (more details are given in the next subsection).
After the MAKECMA has completed its work, the control is handed back to
the CNTOBSPR. Again, if it is a parallel run subroutine POSTPROC_MPP_MAKECMA
(see Fig. 1.3 ) is called, otherwise the
task is about to finish. The main purpose of the POSTPROC_MPP_MAKECMA subroutine
is the following. The CMA data, and for that matter their BUFR counterparts,
are scattered across PEs in a semi-random way. This may be a reasonable
distribution to start with when we know very little about the input data.
However, now that we know much more about the input data and, by combining
that knowledge and our knowledge of what work is going to be done in the
IFS with the data, we are in a better position to do an additional re-distribution
of the input data with a view to getting a reasonable load balance within
the IFS. It is via this subroutine, and the subsequent calls to the other
relevant OBSORT subroutines, that a shuffle and eventual write out of both
the CMA and the initial feedback BUFR data is done. The number of files
written out will either be a single one, or one per PE. Which of this two
options is taken depends on the status of the NAMRUN namelist switch LCOLLECT.
The flow diagram of this subroutine is shown in Fig. 3.1 . The main purpose of this subroutine
is threefold:
|
1) to initialize various MAKECMA task parameters
and switches, |
|
2) to print out the chosen run set-up, and |
|
3) to invoke either the single task (MKCMASIN)
or the parallel (MKCMAMPP) data processing. |
Figure 3.1 Subroutine MAKEMA flow diagram
The setting of the parameters and switches within the initialization consists
of defining:
| |
• numerical limits (SULIM), |
| |
• variables' numbers (SUVNMB), |
| |
• level/layers structure (SULEVLAY), |
| |
• CMA observation and code types (SUCMOCTP), |
| |
• BUFR observation types and subtypes (SUBUOCTP), |
| |
• observation events (SUEVENTS), |
| |
• observation flags (SUFLTXT), |
| |
• various observation processing codes
(SUCODES), |
| |
• BUFR data structure and format (SETBUFR), |
| |
• SIMULATED data structure and format, |
| |
• CMA data structure and format (SUCMA), |
| |
• observation error specification (SUERRORS), |
| |
• valid satellite ids (SUSATID), |
| |
• satellite retrieval types (SUSATRET), |
| |
• SCATTEROMETR observation processing options
(INIERSCA), |
| |
• SSMI observation processing options (INISSMIP),
and |
| |
• appropriate setting for BUFR software
(BUPRQ). |
Once the OBSPROC is initialized, the chosen run set-up is printed (PREAMB).
The next step, depending on whether the parallel or single-task option is
chosen, is to call either subroutine MKCMAMPP or MKCMASIN, respectively.
There is not much difference between these two subroutines. The main difference
is in the way the observation sequence number is handled. MKCMAMPP flow
diagram is shown in Figure 3.2.
The observation sequence number is a unique number assigned
to each observation. This number, in the single task case is simply incremented
(starting from 1), whereas in the parallel case a list is created. Observation
sequence number list creation is independent of how many PEs are used. This
is a very important point with respect to achieving computational reproducibility
later on in the IFS.
However, both of these two subroutines, after some initial preset, open
up input/output files. BUFR input and output files are initialised by calling
INIIBUFF and INIOBUFF subroutines, whereas the CMA file is initialised via
a call to INICMAIO subroutine. BUFR debugging/diagnostic files are initialised
via calls to INIDEBDE, INIDEBEN and INIDEBRE subroutines. Also, a preset
of various counters and pointers takes place.
The initialization of all files, except the input BUFR, consists of simply
opening them up. The decision on which BUFR input files are to be initialized
is controlled by a set of logical NAMBUFR namelist switches. These switches
are: LCONVF, LTOVSF, LATOVF, LSCATF, LSSMIF, LGEOSF and LAUXBF. They control
the presence of input BUFR files for: CONVENTIONAL, TOVS, ATOVS, SCATTEROMETER,
GEOSTATIONARY, SSMI and any other BUFR data. The decision on how many input
BUFR files are to be initialized at this stage, depends on whether the single-task
or parallel version is used. In the-single task version it may be up to
7 files per data time window. However, in the parallel case the input BUFR
files would have already been read in and data distributed across the PEs.
As mentioned earlier the distribution of the input BUFR files is such that
there is one BUFR file per PE containing all observation types. Due to this,
the original BUFR files presence switches are turned off except for the
BUFR auxiliary file LAUXBF. Regardless which way was used to reach this
point, the INIIBUFF subroutine is called. A BUFR input file is first opened
up and its data descriptor record (DDR) read in. If the DDR is present,
the file date and time are extracted and checked against the analysis date
and time. However, if the DDR is not present the very first BUFR message
is read in and the date and time are taken for the date/time checking (allowing
for data time window which is currently  hours). If the date/time check fails further processing
is stopped. Upon a successful date/time check and if the BUFR input data
are to be held in memory (the NAMBUFR namelist parameter LIBUFMER=.T.) subroutine
BUFRLOAD is called to read the whole file in, after which the control is
handed back.
The next step is to create preliminary CMA data descriptors (DDRs). This
is done by calling CMADDR subroutine. After this, the main input BUFR data
cracking subroutine (SCANBUFR) is invoked, and upon its completion the control
is returned back. It is only now that the proper CMA DDRs can be created
by recalling the CMADDR subroutine and linking it up with the rest of the
CMA data (CMA2CMO). Some BUFR and CMA data processing informations are printed
next by calling BUFRSTA and CMASTA subroutines, respectively. The only thing
left to do at this stage is to close off the files. The BUFR input and output
files are closed by calling subroutines FINIBUFF and FINOBUFF, respectively.
The CMA file is closed by calling subroutine FINCMAIO. The debugging/diagnostics
files are closed by calling FINDEBDE, FINDEBEN and FINDEBRE subroutines.
Figure 3.2 Subroutine MKCMAMPP flow diagram
The main purpose of this subroutine, the flow diagram for which is shown
in Figure 3.3, is to loop
over all input BUFR messages. Once it gets hold of a BUFR message it expands
BUFR sections 0, 1 and 2 and hands over further processing to subroutine
OBSCREEN. Subroutine GETIBUFR is used to get a single BUFR message. Depending
on whether the input BUFR data are file or memory resident (the NAMBUFR
namelist parameter LIBUFMER=.F./.T.) data either have been read in, or have
just been copied. BUFR sections 0, 1 and 2 are expanded by calling EXPBUFSC
subroutine. From these BUFR sections a few parameters (BUFR edition number,
observation type and subtype) are taken. The BUFR edition number is checked
and, if it is not expected, further processing is stopped. Also, the BUFR
observation type and subtype are mapped onto the OBSPROC internal numbers
(BUOCTMAP). At this stage, it is possible to skip certain BUFR messages
by consulting the NAMDIA namelist NSKIPBR array. Also, a range of consecutive
BUFR messages can be written out; this is controlled by the NAMDIA namelist
parameters NBURAN1 and NBURAN2. Furthermore, a compressed BUFR message can
be uncompressed by calling BOPRPRO and SPLITCR subroutines. After all this,
further observation processing is handed to the OBSCREEN subroutine.
Figure 3.3 Subroutine SCANBUFR flow diagram
On its arrival to this subroutine (see Figure 3.4), BUFR sections are expanded
again and several BUFR parameters are cross-checked. Also, at this stage
the number of BUFR reports present in a message is found. The BUFR observation
type and subtype are then mapped into the OBSPROC internal numbers by calling
BUOCTMAP subroutine. A selection of BUFR messages which go further is made
here next. This selection is based on examining appropriate observation-type/subtype
switches which are held in the NAMGLP namelist. For a complete list of switches
see the description of the NAMGLP namelist in
Chapter 9 `NAMELISTS' .
The actual expansion of the BUFR messages takes place next by calling the
EXPBUFR subroutine. The expanded BUFR message format is checked against
its predefined template by calling the BUFRFOT subroutine. For some BUFR
land-surface subtypes, there are two valid templates; hence it is difficult
to know which one to use for this type of check. For this reason these subtypes
are checked against both templates, and the subtype changed to an internally
invented one in order to avoid this confusion later on. In the case of the
incoming BUFR message being compressed (more than one BUFR report), a preparation
for its uncompression takes place next by calling PRESPLIT subroutine.
In the next step some preparation is done for possible CMA report creation.
From the BUFR observation type and the subtype, thecorresponding CMA observation
type and code type are found by calling the BUF2CMAT subroutine. The CMA
observation type and code type are then mapped into the OBSPROC internal
sequence numbers by calling CMOCTMAP. Also, the lengths of the CMA header
and body entry are worked out by callingGETHEADL and GETBODYL, respectively.
Now the processing of BUFR reports can start. First, the observation sequence
number is assigned. The way this is done depends, as mentioned before, on
whether it is a single-task or a parallel run. Once, it is established that
a BUFR message is to be processed an appropriate subroutine is called. The
choice of which of the subroutines to call is based on the BUFR observation
type; they are called the basic observation handling routines. Thus, land-surface
data are handled by LANSUIN, sea-surface data by SEASUIN, upper-air sounding
by UPPAIIN, satellite soundings by SATSOIN, aireps by AIREPIN, satobs by
SATOBIN and SATAMIN, scatterometers by SCATSIN, ssmis by SSMISIN and paobs
by PAOBSIN. Further stratification within the above mentioned basic observation
handling routines is possible, based on the observation subtype. The decision
on whether to deepen the processing further is based on how complex an observation
type is. For example, in the case of the AIREPIN routine there is no extra
granulation. On the other hand, in the cases of UPPAIIN, SATSOIN and SEASUIN
there is a specialised subroutine for every possible subtype. Thus, the
land TEMPs are dealt with by LANTEIN, ship TEMPs by SHPTEIN and so on. If
a CMA report has been created, several arrays/pointers are updated first,
and then the CMA report is either written out on a file or added to memory
(ADDCMAR). The NAMCMA namelist switch LOCMAMER is used for this. LOCMAMER=.T.
enables the memory-resident CMA, otherwise the CMA is file resident. At
this stage, if requested via the NAMDIA namelist, both the CMA and the BUFR
report can be printed by calling the RETCMA and RETBUFR subroutines. Since
the expanded BUFR message can be compressed before storing it, a new BUFR
message is created containing just that report. This is done by calling
either the BUCOMP or SPLITCR subroutines. Which of these is called is controlled
by the NAMBUFR namelist switch LBSSPLIT. Furthermore, the BUFR report from
which the CMA report is created can also be either written out on a file
or added in the memory (ADDBUFRR). The NMKCMA namelist switch LIFDBACK controls
whether to write the BUFR report at all, whereas the NAMBUFR namelist switch
LOBUFMER controls whether it would be written out on a file or added to
memory. After all reports from BUFR message have been processed the control
is handed back to the SCANBUFR.
Figure 3.4 Subroutine OBSCREEN flow diagram
As mentioned the above, this group of routines deals essentially with extracting
all the required observations' information from a BUFR report, and with
converting them into the CMA format/structure. The list of subroutines in
this category is:
| |
• LANSUIN (land surface data), |
| |
• SEASUIN (sea surface data): |
| |
• UPPAIIN (upper air soundings): |
| |
• SHPPIIN (ship PILOT), and |
| |
• WINPRIN (wind profiler), |
| |
• SATSOIN (satellite soundings): |
| |
• HIRTOIN (tovs radiances), |
| |
• LLTTSIN (low level temperatures), |
| |
• HLTTSIN (high level temperatures), |
| |
• MERTSIN (merged low/high level temperatures
and pwc), |
| |
• RAD1CIN (level 1C radiances), |
| |
• SCATSIN (scatterometer), |
Because of the complexity of some observation types, their basic observation
handling subroutines are just routines for further branching off, according
to the BUFR observation subtype. Thus, the subroutines SEASUIN, UPPAIIN
and SATSOIN have an additional stratification.
All the basic observation processing routines, whether invoked directly
from the OBSCREEN or from one of the cover routines, have more or less the
same strategy, they:
|
(a) get basic BUFR report parameters, |
|
(b) extract BUFR observation variables, |
|
(c) create a CMA report (the report is created
it two phases: header and body creation phases) |
|
(d) tidy up and hand the control back to the
calling routine. |
As an example, the flow diagram of AIREPIN is shown in Figs. 3.5 and 3.6 .
Extraction of the basic report parameters, such as the coordinates, date/time
and station identification is done by calling subroutine GETREPP. The coordinates
are checked first to see if they are within physical limits and within the
requested geographical area (REPSEL). If this type of check is successful,
the BUFR date and time parameters are converted to the CMA date and time
parameters (OBSDTTM), and they are checked to see if they are within chosen
data-time window (TIMDIF). Furthermore, the BUFR station identification
is converted into the CMA station identification (CHAR2INT). The next step
is to extract quality control flags for the report parameters. Some BUFR
observation types/subtypes are not subjected to the report data-base (RDB)
quality-control procedure, and hence do not have accompanying quality control
information.
Figure 3.5 Subroutine AIREPIN flow diagram
The extraction of the BUFR observation variables is observation-type/subtype
dependent and namelist driven. Generally speaking, the conventional observations
(SYNOP, AIREP, SATOB, DRIBU, TEMP, PILOT and PAOB) carry similar observed
quantities. However, SYNOP may contain much more informations. An important
distinction among the conventional observations is whether they are single-level
(SYNOP, AIREP, SATOB, DRIBU and PAOB) or multi-level observations (TEMP
and PILOT). Effectively, regardless of the observing system, each observed
quantity that needs to be extracted is associated with a specialized routine
to do the task. Also, in addition to the observed value, they extract, if
available, an accompanying RDB quality-control flag. The extraction subroutines
which fall in this group are:
| |
• GETWDF (wind direction and speed), |
| |
• GETPREST (pressure tendency), |
| |
• GETSHPDS (ship speed and direction), |
| |
• GETRADI (brightness temperatures), |
| |
• GETSIGMA0 ( ). |
Figure 3.6 Subroutine AIREPIN flow diagram (continuation)
All of these routines are just cover routines to prepare for the actual
extraction of an observed value and its flag. The actual extraction is done
by calling the GETVLFL subroutine. The decision on whether or not an observed
quantity is extracted is controlled by the NMMKCMA namelist array NBUFRVSE.
This is a three dimensional array of variable number, observation subtype
and observation type. The content of this array is either 1 or 0. If the
value is 1 the quantity will be extracted.
The next step is to create a preliminary CMA report header. In order to
do this, a basic CMA report header is first created by calling the GETHEAD
subroutine. The idea here is to get a correct CMA-report header structure
with the default values set. Secondly, a few report parameters are prepared
for storing into the header. These parameters are the observation characteristics
(OBSCHAR), observation instrument type (OBINSSP) and the conversion factor
for changing coordinates from degrees to radians. Since these parameters
belong to a fixed part of a report header, their storing is carried out
by calling the FILFHDR subroutine. On the other hand, the optional part
of the CMA-report header is observation-type dependent, and there is a specialised
routine for storing those parameters which are particular to a given observation
type. These routines are:
| |
• TOVSOHD (high resolution tovs), |
| |
• TOSAOHD (standard TOVS/SATEM), |
| |
• SCATOHD (scatterometer). |
After both the fixed and the optional parts of the CMA-report header have
been created and filled up as much as possible at this stage, they are added
to the CMA report by calling the ADDHEAD subroutine.
Now comes the creation of the CMA-report body. The CMA-report body consists
of a number of entries. For every single piece of information, provided
it is asked for, an entry is created. The decision on whether or not a piece
of information is to be added is controlled by the NMKCMA namelist array
NMKCMVSE. This is a three-dimensional array of variable number, observation
code type and observation type. The content of this array is either 1 or
0. If the value is 1 then an entry will be created. It is at this stage
that some observed variables are converted to those needed by the analysis.
Also, observation errors are assigned at this stage too. The persistence
error is assigned by calling the OBSPERR subroutine, whereas the prescribed
observation error is assigned by calling the OBSERR subroutine. The actual
observation error (final observation error) is then worked out by calling
the FINOERR subroutine. Once all relevant information is available a body
entry is added to a report. Since a body entry is structured in three parts
(fixed, run-dependent and observation-dependent parts), there is a cover
routtine for each observation type which deals with it. The cover routines
are:
| |
• TOVSBE (high resolution tovs), |
| |
• TOSABE (standard satem/tovs), |
Firstly, each of these routines calls the GETBODY subroutine to get the
correct body-entry structure filled up with default values. Then, they call
the CRECBODE subroutine to finish off the fixed part of the body entry by
calling the FILFBDE subroutine. It is only after this that they call specialized
routines to deal with the observation-dependent part of the body entry.
This specialised routines are:
| |
• CRTOBODE (tovs radiances), |
| |
• CRSCBODE (scatterometer). |
Once the body entry has been created it is added to the CMA report by calling
ADDBODY subroutine. Every time the body entry is created and added to the
report several parameters are updated by calling BODYST subroutine.
Finally, the CMA-report header (fixed part) is updated with several items
of information (e.g. number of body entries) and is replaced back into the
CMA report by recalling the FILFHDR and ADDHEAD subroutines, respectively.
Next Section
Previous Section
|
