HALOE Data at Imperial College: A Users Guide

  1. Introduction
  2. HALOE Data at Imperial College
  3. Example IDL Routines
  4. Further Datasets
  5. Getting More HALOE Data
  6. Further Information

    • 1. Introduction

    This guide is intended to help new users read, analyse (and possibly extend) the HALOE data available at Imperial College. If you need more general information about HALOE, the HALOE homepage provides a wide range, including instrumental details, data quality and validation, a summary of key scientific results and a list of HALOE-based publications. The HALOE homepage at the British Atmospheric Data Centre (BADC) also has useful information and links, whilst the Imperial College HALOE homepage details current HALOE research here.

    Large quantities of HALOE data files for water vapour, methane, ozone and aerosol are available at IC. In this guide Section 2 shows the user how to locate and read these data, whilst details of sample IDL graphics programs to analyse and plot the data appear in Section 3. In addition, Section 4 provides information about some higher level products (such as multi-year mean climatologies, and zonal mean files that can be sampled to produce a time series) that may also prove useful. The HALOE data files currently available at IC are just a subset of the total HALOE dataset, and thus may not match the requirements of new users. Accordingly, Section 5 shows how users can extract more data from the large HALOE dataset stored at BADC.
    Back to top of page

    • 2. HALOE Data at Imperial College

    The HALOE Data Disk, which is located in the subdirectories of /data/haloe on the vulcan Sun workstation (IP address 155.198.199.227), contains both version 17 and version 18 HALOE data (note that the /data disk is accessible from any SPAT workstation). The HALOE retrieval methods have undergone many refinements, and version 18 is the latest version available; it is recommended that new users use the version 18 data.

    Version 18 Data

    Version 18 HALOE data are in directories /data/haloe/v18/haldata and /data/haloe/v18/halunf. The former contains compressed ASCII files, whilst the latter contains unformatted (binary) files. Program writunf in directory /data/haloe can read an (uncompressed) ASCII file and convert it to a binary, and shell script ma enables this to be done for multiple days. The IDL programs described in Section 3 are set up to read binary files. The ASCII files are used less often for large-scale data processing, though they can be useful for checking individual data points.

    Each file contains data for all events (both sunrise and sunset) on one UARS day. There are separate files for each constituent. For each event, there are five profiles: pressure, altitude, temperature, mixing ratio (or aerosol extinction) and quality indicator.

    Currently, the directories contain water vapour, ozone, methane and aerosol (NO channel) data for UARS days 30-2154. The filenames have the form

    hal_d&ltdayno>_&ltsubtyp>_v&ltversno>.dat.1.Z

    (for ASCII data), and

    hal_d&ltdayno>_&ltsubtyp>_v&ltversno>.dat.1.unf

    (for binary data), where

    &ltdayno> is the UARS day number, which equals 1 for UARS launch day, 2 for the second day in space, etc,

    &ltsubtyp> is the variable type: `h2o' for water vapour, `ch4' for methane, etc. Aerosol (NO channel) is denoted by `noaero'.

    &ltversno> is the version number.

    Version 17 data

    Version 17 HALOE constituent data are in directories /data/haloe/haldata and /data/haloe/halunf and the files have the same structure as the version 18 files described above. Currently, the disk contains all version 17 retrievals for water vapour, methane and ozone for UARS days 30-1586. There are also some HF files available.

    The version 17 aerosol data are located in /data/haloe/aero/haldata and /data/haloe/aero/halunf. These files are in a separate directory because the naming convention of the aerosol files is slightly different to the constituent files, and in addition the contents are different. For each event, there are only two profiles: pressure and aerosol extinction. Program writunf in the /data/haloe/aero directory reads an aerosol ASCII file and converts it to a binary. This program indicates the different format used for the aerosol binary files, and thus may be useful if you wish to alter your IDL, or other, read programs.

    Currently, the disk contains aerosol extinctions retrieved from the NO, HCl and CH4 channels, for UARS days 30-1586 (NO), 30-1230 (HCl) and 30-1299 (CH4). The filenames have the form

    hal_d&ltdayno>_&ltsubtyp>_v&ltversno>.dat.Z

    (for ASCII data), and

    hal_d&ltdayno>_&ltsubtyp>_v&ltversno>.unf

    (for binary data), where

    &ltdayno> and &ltversno> are as defined above, and &ltsubtyp>, the variable type, is `hcl', `ch4' and 'no' for aerosol retrievals made by the HCl, CH4 and NO channels, respectively.
    Back to top of page

    • 3. Example IDL Routines

    Much of the reading, analysing and plotting of the HALOE data at IC is done using the IDL graphics language. A number of example programs, plus all the required subroutines, are available in directory /data/haloe/idlprogs on vulcan. These programs are set up to read, interpolate and plot HALOE data that reside in directories /data/haloe/halunf and /data/haloe/v18/halunf. The details are as follows.
    hal_multi.pro
    This reads HALOE data for one day then plots the profiles. Sunrise and sunset data are treated separately. Profiles can appear individually or in multiples in each plot, and the program is set up to produce up to four plots on the one page
    A further two programs read in HALOE data for a specified number of days, interpolate them to fixed pressure levels (between 100 and 0.01 mb, regularly spaced (interval 0.04) in log pressure), and then to to a regular grid with a resolution of 5o latitude by 8o longitude. No time interpolation is carried out.
    hal_zm_plot.pro
    The above field is zonally averaged and a latitude v. pressure plot is produced.
    hal_map_plot.pro
    This program extracts data at selected pressure levels then produces a latitude v. longitude plot, with the option of polar stereographic or cylindrical projections.
    A fourth program, hal_thmap_plot.pro, is similar to hal_map_plot.pro, except that the data are initially interpolated to isentropic surfaces rather than to pressure levels.
    Back to top of page

    • 4.Further Datasets

    The long period of the HALOE dataset means that any new user interested in producing a multi-year climatology or in examining interannual variability may have to perform a lengthy and computationally demanding amount of data processing. There are two sets of processed HALOE data available on mimsey that may be useful to new users, and which may cut down some of this processing (mimsey's IP address is 155.198.199.26. Note that the /disk2d disk (see below) is accessible from any SPAT workstation).

    Multi-Year Climatologies

    A climatology of water vapour (for version 18 retrievals) is available in the /disk2d/david/v183dpmap directory. The fields are three dimensional, and expressed on a 5o latitude by 8o longitude at 108 pressure levels between 190 and 0.01 mb. Potential users of these fields should note that Harries et al ( J. Geophys. Res. (1996), 101 10205-10216) estimated the HALOE water vapour retrievals to have an accuracy of +_ 10 % in the 0.1-100 mb range, rising to +_ 30 % at the boundaries of the observational range. The climatology has been calculated for the period December 1992 - February 1997 (prior to December 1992, the large quantites of Mount Pinatubo aerosol in the lower stratosphere gave rise to very large HALOE retrieval errors).

    The data are written in (46,37,108) element arrays, where the first element is longitude, the second latitude and the third pressure, and can be read using the IDL subroutine p_read_3dfield.pro, which appears in the /data/haloe/idlprogs directory.

    Seasonal mean fields, and multi-year averages of these seasonal means, are available. Both types of field average together both sunrise and sunset data. The filenames are of the form

    v_18_h2o_&ltseason>_&ltyear>_sr+ss_unsmoothed_.dat

    (for individual seasons)

    and

    v_18_h2o_&ltseason>_&ltnumber&gtyr_Average_sr+ss_unsmoothed.dat

    (for multi year seasonal averages), where

    &ltseason> = 'Winter' (for December - January -February), 'Spring' (for March - April - May), 'Summer' (for June - July - August) and 'Autumn' (for September - October - November),

    &ltyear> is '1992_93', '1993_94', etc for the Winter season, and '1993', '1994', etc for the other three seasons, and

    &ltnumber> = 5 for Winter averages, 4 for other seasons (since there are five Winter seasons in the December 1992 - February 1997 period, and only four of the other seasons).

    The missing data flag is 9 x 10-6 for individual seasons and 999 for the multi-year means.

    There is also a climatology of ozone available. The ozone fields fields have similar names to the water vapour fields, except that v_18_h2o at the front of the filename is replaced by aersm_v_18_o3. This is because the ozone retrievals are very highly sensitive to high aerosol amounts, and thus the ozone climatology excludes all retrievals at points where the retrieved aerosol extinction exceeds 5 x 10-3 km-1. This means that at locations near the tropical tropopause, the climatology is over-smoothed in the zonal direction, and thus should be used with caution.

    Zonal Mean Fields for Time Series

    Zonal mean fields of HALOE data have been calculated for the entire duration of the HALOE dataset using the hal_zm_plot.pro IDL routine. Sunrise and sunset data are kept separate. Three categories are available:
    `0o Centred'
    Each time HALOE observes in the 40oS-40oN region, data from all days in such a period are used to create a gridded field between 35oS and 35o.
    `30oN Centred'
    Each time HALOE observes in the 10oS-70oN region, data from all days in such a period are used to create a gridded field between 5o and 65o
    `30oS Centred'
    Each time HALOE observes in the 70oS-10oN region, data from all days in such a period are used to create a gridded field between 65oS and 5oN.
    By sampling a succession of such fields at any pressure or latitude, one can quickly create a time / pressure or time / latitude section at the required location. The `0o Centred' category data are best suited for examining low latitude temporal variability, whilst the other two are better for mid latitude variability.

    The fields are located in directories /disk2d/david/haloe/zero (for `0o centred'), /disk2d/david/haloe/thirty (for `30oN centred'), and /disk2d/david/haloe/nthirty (for `30oS centred'). Subdirectories of /disk2d/david/haloe called thzero, ththirty and thnthirty are like the above, except they contain theta / latitude fields instead of pressure / latitude fields. Fields are available chiefly for water vapour and temperature, but also sometimes for ozone, methane and total hydrogen (equal to 2CH4+H2O). Check the README files in the above directories for further details. The filenames have the form

    v_&ltversno>_&ltsubtyp>_&ltdayno>_&ltevtyp>.dat

    where

    &ltversno> is the version number, either 17 or 18,

    &ltsubtyp> is the variable type: `h2o' for water vapour, `ch4' for methane, `o3' for ozone, `D' for total hydrogen and `temp' for temperature,

    &ltdayno> is the mean UARS day number of the time period of data selected to calculate the zonal mean field, and

    &ltevtyp> is the event type, `sr' for sunrise data and `ss' for sunset data.

    The p_read_zmfield.pro subroutine in directory /data/haloe/idlprogs can be used to read the zonal mean fields located in the subdirectories of /disk2d/david/haloe.
    Back to top of page

    • 5. Getting More HALOE Data

    Sections 2-4 deal with reading and manipulating HALOE data that are already available at IC. But what if you need more HALOE data? This section details how to go about extracting HALOE data from BADC.

    The HALOE data files at BADC are called Level 2 files, and they contain vertical profiles of retrieved temperatures, mixing ratios and aerosol extinctions expressed at the appropriate HALOE observational latitudes and longitudes. The Level 2 data are organised into one file per day, and as well as containing retrievals from all the HALOE channels they contain a lot of detailed information on the instrument and its performance. This means that these files are very large, approximately 14 Mbyte for each daily file. The programs detailed below extract the required retrievals from the Level 2 files, and then these files (which are created separately for each day and each constituent) can be FTPed back to IC. Since these files do not contain any ancillary instrumental information they are much smaller than the Level 2 files (for example, water vapour files are around 0.3 Mbyte per day), and can thus be much more easily stored and processed.

    The HALOE data are on the tornado machine at BADC ( tornado.badc.rl.ac.uk). The BADC Contact Page gives information on who to contact about getting an account on tornado. Since HALOE is a restricted access dataset - BADC must obtain permission from the HALOE Principal Investigator before they can give anyone access to the data - users must also complete a `Conditions of Use' form before they can use the data.

    Directory /data/haloe/badcsoftware on vulcan contains example software that can be run on tornado to extract daily files of HALOE water vapour, ozone, methane and aerosol (NO channel) from the Level 2 data files at BADC. Copies of much of the software required are available on tornado in directory /badc/haloel2/software/examples. The programs shown here are modifications of these. They run in directory /data/tornado/djackson/read18 on tornado . Details are as follows.

    read_h2o.f
    Main program to extract water vapour data from the Level 2 file. The output is an ASCII file similar to those in directory /data/haloe/v18/haldata. Files read_o3.f, read_ch4.f and read_no_aero.f are similar, but instead extract ozone, methane and aerosol (NO) data, respectively.
    mf.unix
    Makefile to compile read_h2o.f ( mf_o3.unix, mf_ch4.unix, etc compile read_o3.f, read_ch4.f, etc). The other files you require (the *.inc files, rl2dat.f, rl2sum.f, rwl1hd.f) are available from directory /badc/haloel2/software/examples on tornado. You do the compilation by typing make -f mf.unix. (NB: the line of the form f77 -convert big_endian -c read_h2o.f was commented out of the make file due to some insoluble problem. After performing the make -f mf.unix command, you have to type in the above line to ensure that the read_h2o.f (or other) program is updated).
    read18run
    This is a shell script set up to extract data from Level 2 files for a specified number of days The example here runs between UARS days 503 and 510 and produces ozone files of the form hal_d0&ltday>_o3_v18.dat.1, where &ltday> is the UARS day number, from HALOE Level 2 files located in (tornado) directory /badc/haloel2/data/1993. Note that here the Level 2 filename ends in c01_rac. Some of the filenames end with prod rather than rac, and also may use c02, or higher numbers, rather than c01. Therefore, to include all data within a specified period, you may have to re-run with an altered filename ending, or use a wildcard.
    Back to top of page

    • 6. Further Information

    For further information about the data and programs discussed here, contact Adam Hicks. More general computing enquiries should be addressed to the Group Computer Manager Graeme Willoughby .Back to top of page
    Back to HALOE homepage.
    Back to SPAT home page. 
    Last modified 26/01/1999.
    WWW page comments to Adam Hicks