Interface File Internal Syntax

Each line will have an identifying first character followed by a string of appropriate syntax and terminated by a new-line character (` n' in C, any carriage return characters will be ignored).

comments

These will take the form:
% text string
and are intended to help make the file more readable.

messages

These will take the form:
M text string
and are intended to provide a mechanism for identifying information that may be caught by a filter for display on the console and passed to a log file for operator inspection. For example, warning messages that are normally displayed during data reduction should also be copied to the ``.iff'' file to ensure that such information is available during later CDF file generation and short term archive.

fatal error

These provide error state communication to the CDF writing software from the other components of the pipeline data reduction software. They will take the form:
F text string
where the text message is for operator use in investigating the cause of the error. These differ from `messages' in the sense that they are for communicating fatal error states rather than warning messages. They will cause termination of the processing on the active interface file, and the resulting cdf file will be deleted by the CDF writing software to ensure that other automatic/operator processes do not distribute corrupted data files. The text message will be written into the log file.

global attributes

These will take the form:
G Attribute _name Value
where `Attribute_name' is the CSDS defined name for the attribute (a list is provided in appendix A) and `value' is the entry to be placed in the attribute. The CDHF software will keep track of multiple entries as appropriate. If `value' is a text string it may either be delimited by white space or a double quote, ". If the text string itself contains white space (e.g. sentences used in MODS) then it must be delimited by double quotes. For CSDS files the data type of a global attribute entry is the same as that in the skeleton file.

Each entry of a global attribute of char data type contains a single string, but without the null terminator. Global attributes of other data types may have more than one element per entry. At present multiple elements are only necessary for the attribute `Data_intervals', which is constructed by the CDHF CDF writing software, and is not required in the .iff files. Nevertheless, a facility is provided for general processing, and against future evolution of CSDS CDF file design. Thus if multiple values are supplied in the global attribute line in the .iff file e.g.
G Attribute_name Value1 Value2 Value3
where Valuen are data types other than char strings, they will be entered as successive elements in a single global attribute entry.

variable attributes

These will take the form:
V Variable_name Attribute_name Value(s)
where `Variable_name' is the name of the CSDS data product to which it applies, `Attribute_name' is the CSDS defined name for the attribute (a list will be provided as an appendix) and `value' is the entry to be placed in the attribute. Variable attributes may take multi-element entries, and so `value' may be a space separated list of elements for any data type other than character. Multi-element character entries are treated as text strings. If `value' is a text string it may either be delimited by white space or a double quote, ". If the text string itself contains white space (e.g. sentences used in MODS) then it must be delimited by double quotes.

data records

These will take the form:
R Record _number UTC_time_tag millisecs_time_tag dependent_variables
where `Record_number' is a sequential numeric label starting at zero. This is to aid in system tests. The entry `UTC_time_tag' is the UTC time for the measurements in that record; `millisecs_time_tag' gives the time converted to milliseconds past midnight of the previous day. `Dependent_variables' is a list of values for each variable in the order specified in the instrument specific appendices.

Note that the `UTC_time_tag' is provided only to improve the readibility of the file. The actual time used within the CDF will be taken from the `millisecs_time_tag' entry (to calculate the Epoch variable).

Each entry in the record is separated by at least one space character.

Any variable of data type `character' must be delimited by double quotes since they may contain white space or other characters. Character type variables may be multiple entry and are stored as text strings in the CDF file, they are entered in the .iff file as quoted strings (e.g. "A %+"). Since the Status variables have become arrays of 1-byte integers, there are no character type variables for any of the UK led instruments in the CSDS data base. This facility is, however, retained against future evolution of the CSDS data products.

Dependent variables must be supplied in the same order that they appear in the appropriate skeleton files. This is the same as the order used in the instrument-specific appendices, where sample files are provided. Elements of GSE vectors must appear in the `natural' order, x,y,z, separated by spaces. No PPD or SPD data products have higher dimensionality than vector, but the CDHF supplied code will be able to write variables of any dimensionality and size into CSDS compliant CDF files. When used for these non-CSDS files with higher dimension data, the order of array elements should be the natural order in C programs, that is the last index varies the fastest (e.g. A[0][0] \ A[0][1] A[1][0] A[1][1] for a 2x2 array) with elements separated by at least one white space character. The input record length is limited to 400 characters as this is sufficient for all UK CSDS data files, but this limit may readily be increased to accomodate input of large data arrays.

Data Gaps

These are identified by a line with the character N in the first column. Any number of gap lines are allowed, and constitute a single gap in the data between the record preceding the gap line(s) and the record following it. This information is used by the CDF creation software to populate the attribute `Data_intervals' which identifies intervals of continous data.

End of Processing

The end of processing marker will be the character `E' in the first column followed by a white space character and the number of data records written into the file as an integer. This latter is provided as a consistency check. Any data record entries beyond this end of file marker will not be placed in the CDF file, and a warning will be written into the logfile. Absence of this flag within a .iff file will be treated as a fatal error state. This marker implies the successful completion of the data reduction processing. The end of processing, `E', marker and the fatal error, `F', marker are mutually exclusive.

Fill Values

An optional line containing fill values in the same format as a data record may be placed anywhere in the file. It differs from a `record' line in that the first character is `U' in place of `R'. This line will be interpreted as the values to be expected for each variable in a record line if they cannot be provided by the processing software. When these values are encountered for the respective variables in a record, the CSDS standard value FILLVAL for that data type will be written in their place into the CDF file. If this line is not present in the file, then all values will be written into the CDF file unaltered, and fill values used must conform to those specified in the CSDS CDF Reference Document, DS-QMW-TN-0003.

Note that it is intended that this line may be written by the same function call that writes the data records to ensure exact matching of sample fill values and those encountered in records. To enable this, the fill value line must have a dummy record number and time text string as place holders, see examples in appendices. Similarly, if hexadecimal escapes are used for numeric byte data type values in a record line the hexadecimal escape must be used for the fill value as these are detected using string comparisons between the `U' entries and the `R' entries.

Only one fill value line is required per interface file, and this may appear anywhere before the `End_of_data' marker. If multiple fill value lines are encountered, the last such line will be used.