QSAS: The QMUL Science Analysis System

Analysis

# Contents

- Overview
- Arithmetic
- Calculator
- Units Management
- Time Operations
- Plug-ins
- Tips/FAQ
- Bugs

# Overview

QSAS enables the user to perform various kinds of analysis and manipulation of data. In all cases, the operations are performed on objects held on the Working List, and the results of the analysis are, in general, returned as new objects on the Working List from which they may be further manipulated, plotted, or exported.

The main types of manipulations in QSAS are:

1. General arithmetic operations, including +, -, x, /, and various vector operations. All the basic unary and binary arithmetic operations are supported as well as basic trigonometric functions and their inverses. These operations are accessed through the Analysis/Simple Maths pulldown menu on the QSAS Main Window.
2. Advanced arithmetic operations allow the user to combine as many data objects through unary and binary operators to create new objects in the Working List. This is done using the Calculator interface accessible from the Analysis menu item.
3. Units of QSAS objects can be changed, doing the specific conversions where needed. Units can be changed to SI, to match another object's units, or to be converted to any arbitrary unit. The items in Analysis/Units allow these manipulations.
4. Time Manipulation routines such as joining time series onto a common set of time tags which may be those attached to one of the input objects, a different time series object or a set of regular time tags specified within the join interface. Also provided are time shift and stretch operations to move and distort the time lines of a time series object for comparison between spacecraft separated by a time lag or passing through evolving structures. These operations are accessed through the Time pulldown menu on the QSAS Main Window.
5. Plug-ins, which enable user-written routines to be loaded dynamically in order to perform analysis functions. The QSAS team provides an increasing suite of plug-ins, some of which include specialised graphics. These operations are accessed through the Plug-Ins pulldown menu on the QSAS Main Window. The plug-ins distributed with QSAS are grouped via sub-menus into those dealing with geophysical operations, such as coordinate frame rotation and vector manipulation, boundary normal determinations and field models, and scientific analysis such as power specrtum analysis, wavelet analysis and minimum variance routines. A Browse option under the Plug-Ins pulldown menu launches a file selection window to allow the user to navigate to their own plug-in template file.

# Arithmetic

The Analysis/Simple Maths menu on the Main Window of QSAS provides direct access to a number of unary and binary arithmetic operations and vector products. These use the QSAS Plug-in interface to open a window with slots for the input data objects and names of the output object(s) as shown. The Help button provides some detailed information. A new window is opened each time an arithmetic option is selected from the menu so multiple operations can be repeated with different inputs. The arithmetic input slots accept any type of numeric data input.

Input Arguments can be objects from the Working List or, in some cases, floating point constants typed directly in the input slot. Most binary operations on time series data require the input objects to be joined onto the same timetags, and refuse the operation if they are not. Binary operations involving a data/time series and a constant value will apply that constant value to each element of the data/time series; a warning is written to remind the user that this is the assumed behaviour. Constant vectors can be created using the New Object option in the Edit menu on QSAS Main Window.

Output objects are created and placed on the Working List with the name(s) given in the slot(s). These output objects include Units (text strings) and SI_conversion (formatted information) appropriate to the operation. Thus, in the example shown, the object Product will have Units = "km/s /cm^3" (straightforward concatenation of the two Units attributes) and an SI_conversion = "1.e+9>m^-2 s^-1" meaning multiply the values by 1.e9 to convert them into the SI standard units. Other operations, such as addition, are checked for compatible SI_conversions and warnings given if they do not appear to be appropriate.

QSAS chooses the precise operation depending on the input arguments. Thus the same interface is used to multiply a vector time series by a numeric constant (input as 2.0 , or just 2) , or multiply two scalar time series, as in the example, or multiply a vector by a rotationj matrix (either a single matrix or a series of matrices on the same time tags). It is safe to experiment with arithmetic operations as conformality of input objects is checked at run time and a warning issued if the result cannot be varified (such as metadata being unavailable) or the operation is rejected if the result is invalid or ambigupous (such as adding two objects with different SI_conversion values, or multiplying two vectors - for this you must use the explicit vector and scalar products provided in the Maths menu.

# Calculator

See the QSAS Calculator documentation.

# Units Management

QSAS can convert an object's units in whatever unit makes sense using the SI_conversion attribute of the object. The Analysis/Change Units menu offers three ways of changing the units of an object:

- ToBaseSi takes a simple object as input, and converts it to the base SI unit as defined in its SI_conversion attribute. Its DELTA_PLUS, DELTA_MINUS, SCALEMIN, SCALEMAX, VALIDMIN, VALIDMAX and FILLVAL attributes are also converted for consistency. The resulting object is placed as named in the ToBaseSI interface on the Working List.
- ToObjUnits creates a new object on the working list, computed by converting the first input object's units into the second input object's ones. The same attributes as previously are also converted.
- ToUnits creates a new object on the working list, resulting from the conversion to an input object to an arbitrary unit, defined as an SI_conversion attribute. e.g. : specifying 1.0e-3 > A as SI conversion and mA as Unit name converts the input object's data to milli-Amperes if its base SI unit is A.

# Time Operations

## Joining Time Series

Analysis of time-ordered datasets often requires data to be placed onto a common set of time tags prior to further analysis. For example, taking the difference between two time series of plasma density, e.g. n1(ti) and n2(tj) requires that each time ti at which n1 has a value has a matching tj so that n1(ti) - n2(ti) can be found. Since often n1 and n2 come from different sources, this is not usually the case, and one or both of the series n1 and n2 must be interpolated, averaged or otherwise manipulated so that new series n1* and n2* can be differenced. This matter is critical and complex, and the interested reader may wish to consult Chapter 2 (Time Series Resampling Methods by Harvey and Schwartz) in the ISSI book Analysis Methods for Multi-spacecraft Data published by ESA/ISSI and available in electronic form at the ISSI web-site. Generally, when the target timetags are widely spaced with respect to the original set, simple boxcar averaging (with a boxcar twice as wide as the target timetag spacing) is appropriate. For closely-spaced target timetags, linear interpolation is often as good as more sophisticated techniques. In all cases, gaps in the time series need to be trapped and either filled or entries removed.

The QSAS Join routine, accessible from the Analysis/Time Ops.. menu on the Main Window of QSAS, enables the user to control all these aspects of joining time series. All objects to be joined are placed into the slots provided in the upper left portion. These need not be all the same type, e.g., they could be a mixture of scalar, vector, and array time series. The source of the target timetags is specified in the lower right portion. This can be one of the input objects, but could be another time series (i.e., any data object which has a set of corresponding timetags), and an option to create a set of regular time tags with specified spacing is also provided (which are needed if Fourier methods are to be employed).

The resulting objects will all be given the same suffix ending to their names on the Working List.

The upper right panel enables the user to select the interpolation/averaging options and gap treatment, and to specify corresponding parameters. The options chosen apply to the highlighted object(s) on the join object list, and different options may be set for different input objects or groups of objects.

Like the Working List, it is possible to group objects into folders on the join list for convenience if many objects are to be joined at once.

Note that selecting the Fill Value option for filling gaps may result in data series of different lengths if the 'Strip Fill Values' toggle is set on in the QSAS main window. Objects joined with this gap handling choice may therefore fail future tests on whether the data are joined even after they have been joined.

Similar behaviour will result if a zero fill is selected for gap handling if the fill value (held in the Fillval attribute for the object) is also set to zero.

## Time Series Subset, Time Shift and Time Stretch

Data objects from different sources may need to be "synchronised" to reflect the propagation of a physical phenomenon for example. The Time Shift and Time Stretch options in the Analysis/Time Ops... menu allow these changes. The first one takes as an input a time series objects and am offset in seconds, and produces an object on the working list that corresponds to the input object with shifted time tags. The second option takes 3 inputs: the time series whose time tags need to be shifted and stretched, a reference time interval, which can be the object's time interval itself, and a new interval, onto which the reference interval is mapped.

The Time Series Subset option works as a filter, removing from an input object the data not fitting in a specified time interval, and outputting the result as a new object on the Working List.

# Plug-ins

Access to user-written plug-in routines is provided through the Browse option in Plug-Ins pull-down menu on the Main Window of QSAS. A file selection window enables the user to locate the template (*.qtpl) file corresponding to the desired plug-in. The QSAS supplied standard plug-ins are accessible directly through the Analysis and Geophysics sub-menus of the Plug-Ins menu. Currently supplied plug-ins include a suite of magnetospheric coordinate transformations, minimum variance and spectral analysis routines, time-lagged cross-correlations, and a cluster locator, most of which can be done directly using the Qdos C++ methods but are often useful as stand-alone plug-ins.

Use of a plug-in is identical to the use of Arithmetic described above (indeed, all the Arithmetic routines are actually written as plug-ins, and provide useful examples for users wishing to write their own plug-ins).

The QSAS Plug-in interface allows user-written routines to be compiled independently from QSAS. Writing Plug-ins is described in separate QSAS documentation. Briefly, a plug-in consists of a C++ wrapper routine, which accepts the input arguments from the Working List and returns the Output objects to the Working List. This wrapper in turn passes the inputs to the user's routine and collects the results. The user's routine may draw on the C++ methods of the Qdos objects to perform many tasks directly on the input objects (such as addition, masking, joining...) or may unpack the object to pass simple arrays to routines written in C, Fortran, or some other language. Indeed, since control is passed from QSAS to the plug-in, a plug-in may in turn display its own graphical user interface, do its own plotting, or launch an independent package. Finally, an ASCII template file tells QSAS information about the input and output arguments, from which the initial Graphical User Interface is constructed and presented to the user.

Other Plug-ins will be added, and users are encoured to submit their own routines for wider distribution (or requests for other plug-ins they would like to have!). The Refresh Menu item in the Plug-ins menu will reread the Analysis and Geophysics directories to update the list of Plug-ins available through the pull down menus, and this allows newly added plug-ins to become immediately available to an already running QSAS session without restart. Similarly, if a .qtpl template file has been removed from the Analysis or Geophysics directories the corresponding menu item will be removed by the Refresh Menu action.

As plug-ins are loaded dynamically and given programme control, some care is required to ensure that plug-ins are robust against improper inputs. If a plug-in crashes, it crashes the rest of QSAS with it.

# Tips/FAQ

- Constant vectors and other values needed as inputs for analysis routines can be created using the New Object menu item under the Edit Menu on the QSAS Main Window. Double-clicking (browsing) the resulting constant enables the user to add new cross references to specify the UNITS, SI_conversion, and other attributes which might be desirable.
- Always check the text box of analysis routines to verify that the operation was performed successfully and read any warnings issued to ensure that the desired operation was actually performed. Double-clicking the Output product on the Working List will enable you to browse individual values together with the object's attributes (which can be edited if desired by double-clicking the attribute name).
- If you want to do the same sequence of operations more than once (e.g., calculate P = nk(Te+Tp) ) it will pay you to write your own plug-in.
- When writing plug-ins it is advisable to use the save session option in the File menu of the QSAS Main Window prior to launching the plug-in for testing to avoid repeating preliminary operations like data import and join. The restore session can then be used to return QSAS to the point before launching the plugin should the plug-in crash or produce unexpected behaviour. QSAS itself can then be used as a test rig for the plug-in, and the browse and plot capabilities used for testing value returns.
- Note that QSAS is compiled with debug flags on, but these symbols are stripped from QSAS before release to keep the executable smaller. Removing the STRIP_DBG setting from the build script will therefore produce a (large) qsas executable that can be used with a debugger. If your installation was from a binary copy of the qsas executable, ask csc_support for a binary without debug symbols stripped out if you need to use a debugger to go into your executable at run time. It will not, in general, be possible to set break points inside your executable until after the plugin has been loaded (after the Run button is pushed).
- When joining data series with gaps, care must be taken if gaps are filled with zeros or fill values. The Strip Fill Values toggle on the QSAS main window must be unchecked if the gaps are being filled with the Fill Value set in the Fillval attribute. If, after joining, data series fail a test on being joined, then the most likely cause is that values (either Fill Values or NaN) are being stripped as the joined objects are placed on the Working List.

# Bugs

- Joining time series at present will not warn the user if the two time intervals are entirely disjoint, mostly gaps or only partially overlapping. This can result in the data for one series containing very poor data or zero filled or fill values depending on the gap handling choice. It is recommended that data sets are plotted on a common timeline first to determine their suitability for joining.
- There is at present no mechanism to chain analysis operations, nor to save a sequence of operations, but each plugin can be kept open to allow repeating the same sequence without relaunching the plug-ins. Frequently-needed sequences can be incorporated easily into a single plug-in.

Page created by Steve Schwartz, csc-support-dl@imperial.ac.uk

Last up-dated: October 2008 A Rochel