next up previous contents
Next: Sample CEF Up: File Format Previous: Variable Metadata Summary   Contents

Data Records

Data is record oriented with an end of record marker (by default the newline character) denoting the end of each record. The entries for each variable in a record must appear in the order specified by the header descriptors.

Each entry within a record must be separated from its neighbours by a comma (","). Delimiters must be separated by valid input. Missing entries must be padded by a fill value, and the `FILLVAL' metadata value should be used to identify the fill value used. The value of this fill is left to the discretion of the team generating the data.

Each variable may be multi-dimensional with the number of entries per variable being the product of the dimensionalities. Thus a velocity vector has three entries and a two by two array has four entries, while a sixteen by eight by twenty four array has 3072 entries. It must be recognised that some data products may result in extremely long record lines. The rigid record structure to the data is intended to allow simple generic software to handle all instrument files.

The ordering for arrays within a record is the natural `C' ordering, that is, the last index varies the fastest.

Setting an `END_OF_RECORD' character other than the default allows long records to be split across lines with new line characters for human readability. This also allows for software systems with maximum line lengths. The END_OF_RECORD character cannot be either the exclamation mark or the ampersand.

On read all new line and carriage return characters together with white space (including tabs) and the end or record marker are deleted. These characters and non-printing characters are thus not allowed values for character data types.


next up previous contents
Next: Sample CEF Up: File Format Previous: Variable Metadata Summary   Contents
Anthony Allen 2009-10-19