Warning, /graphics/kst-plot/docbook/kst/data-chapter.docbook is written in an unsupported language. File is not indexed.

 <chapter id="workingwithdata">
 <title>Working With Data</title>
 
 <sect1 id="datasources">
 <title>Data Sources</title>
 
 <sect2 id="DataSourceConcepts">
 <title>Data Source Concepts</title>
 <para>
 A data source in &kst; is simply a supported data file.
 Currently, Kst supports ASCII text files,
 <ulink url="http://getdata.sourceforge.net/">dirfiles</ulink>, and
 <ulink url="http://www.unidata.ucar.edu/software/netcdf/">NetCDF</ulink>,
 for vectors and scalars, and FITS images, BIT image streams,
 16 bit TIFF images, and any image format supported by 
 <ulink url="http://doc.qt.io/qt-5/qimage.html">QImage</ulink>,
  (jpg, png, bmp, etc) for images.
 </para>
 <para>
 
 The following concepts are important in understanding how &kst; works with
 different data sources.
 Some terminology is also introduced in this section.
 </para>
 
 <sect3 id="DefineFields">
 <title>Fields</title>
 <para>
   Data in &kst; are accessed by field names.  A field name can refer to a single scalar or string, to a vector of values from a single sensor,
   or to a matrix.  For example, a column in an ASCII data file can be read in 
   as a vector.  An image in a png file can be read in as a matrix.  
   Datasource readers provide functions for reading and obtaining fields and 
   field names.
 </para>
 </sect3>
 
 <sect3 id="Frames">
 <title>Frames</title>
 <para>
 When reading in a vector from a data source field, data are addressed by their Frame number, not by their sample number.  Each field in a data source has its own fixed number of samples per frame.
 </para>
 <para>
 For some data sources (eg, ASCII files) every frame contains exactly one sample (ie, for ASCII files, a frame is a valid row of data, and every row has exactly one sample for each field).  
 </para>
 <para>
 However, for other data sources (eg, dirfiles), there may be multiple samples per frame.  In the illustration below, the first 3 frames of an imaginary dirfile are shown.  In this particular data file, Field1 has a 1 sample per frame, Field2 has 4 samples per frame, and Field3 has 2 samples per frame.  Every field must have a constant number of samples per frame throughout the file.
 </para>
 <para>
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Diagram-kst-frames.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
 </para>
 
 <para>
   In the figure, imagine that time proceeds from top to bottom.  &kst; assumes that the first sample in a frame is simultaneous for every field in the data source, and that the rest of the samples are sampled evenly throughout the frame, as shown.  
 </para>
 <para>  
   When plotting one vector against another, &kst; assumes that the first and last samples of each vector are simultaneous, and interpolates the shorter vector up to the resolution of the longer vector.  Since only the first sample in a frame can be assumed to be simultaneous across fields, <emphasis>when &kst; reads data into a vector, it only reads up to the first sample of the last frame requested,</emphasis> so that plotting one vector against another will make sense.  The rest of the last frame will not be read.  
 </para>
 <para>
   So if the first three frames of Field1 and Field2 are read from the data source in the figure, 3 samples will be read from Field1, and 9 samples will be read from Field2 (ending at first sample of Frame 3) - not 12 as one might expect.
 </para>
 </sect3>
 
 <sect3 id="supportingadditionalfileformatsdatasourceconceptsindex">
 <title>INDEX Field</title>
 <para>
 As well as the explicit data fields in a data file, &kst; implicitly creates an INDEX field for all data sources.  The INDEX field is 1 sample per frame, and simply contains integers from 0 to N-1, where N is the number of frames in the data file.  It is common to plot vectors against INDEX.  This is convenient since  the INDEX of a sample or event is just the frame number, allowing easy identification and retrieval of events from a data file.
 </para>
 </sect3>
 
 </sect2>
 
 <sect2 id="creatingascii">
 <title>ASCII Input Files</title>
 <para>
 &kst; is capable of reading vectors from a wide range of ASCII formats.  As long as the data are in columns, and as long as each non-comment row has the same number of columns, &kst; can probably read it.
 </para>
 <para>
   Consider reading this simple ASCII csv file:  each comma separated column represents a field. 
 </para>
 
 <informalexample>
 <screen>
 Length,Width
 m,m
 1.1,6.2
 2.4,9.3
 4.3,4.7
 5.2,8.8
 </screen>
 </informalexample>
 
 
 <para>
   When you enter an ascii source into a data source selection widget (such as on the first page of the data wizard) the file will be identified as an ASCII file, and the <guibutton>Configure</guibutton> button will be enabled, as shown below.
 </para>
 
 
 <para>
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-datasource-selector.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
 </para>
 
 <para>
   Clicking on <guibutton>Configure</guibutton> will bring up the ASCII data source configuration dialog.
 </para>
 
 <para>
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-ascii-config.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
 </para>
 
 <para>
   Note that the first few lines of the file are shown.  The dialog in the screen shot has been filled out to read this file: looking at the first lines of the file, we see that data starts at line 3, line 1 holds the field names, and line 2 holds the units (which will be used by &kst; in plot labels).  Additionally, as this is a csv file, a "<literal>,</literal>" has been selected as the <guilabel>Custom delimiter</guilabel>.  Selecting <guilabel>OK</guilabel> will assign this configuration to this file.  &kst; will continue to use this configuration with this file until the configuration options are changed again in this dialog, or until <guimenuitem>Clear datasource settings</guimenuitem> in the <guimenu>Settings</guimenu> menu is selected.
 </para>
 
 <para>
 </para>
 
 </sect2>
 </sect1>
 
 <sect1 id="datamanager">
 <title>The Data Manager</title>
 <para>
 The Data Manager provides a central location for deleting and modifying all the data objects used in &kst;.
 It can be accessed by selecting <guimenuitem>Data Manager</guimenuitem> from the <guimenu>Tools</guimenu> menu or by clicking the  
 
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Icon-kst-datamanager.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
 
 icon in the tool bar.
 </para>
 
 <screenshot>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-datamanager.png" format="PNG"/>
 </imageobject>
 </mediaobject>
 </screenshot>
 
 <para>
   The window lists all data objects currently loaded in kst, as well as their type and basic properties.  To edit an object, double click on it, or select it and click the <guibutton>Edit</guibutton> button.  It will bring up the appropriate edit dialog.  Items that have child-vectors can be expanded to list them.  Items also have right button context menus with common tasks for the items, such as making curves or power spectra from vectors, or adding curves to plots.
 </para>
 
 <para>
   Items can also be deleted from this dialog by selecting them and clicking <guibutton>Delete</guibutton>.  This will delete the selected item and all items which depend on it.  So if you delete a vector, all curves, spectra, histograms, or equations that depend on it will also be deleted.  Be careful, because this can not be undone.
 </para>
 <para>
   Clicking <guibutton>Purge</guibutton> will remove all undisplayed data objects.  If deleting an object (and everything that depends on it) will not change any plots or labels, then it is deleted.  Nothing that is displayed is deleted.  This can not be undone.
 </para>
 
 </sect1>
 
 <sect1 id="datatypes">
 <title>Data Types</title>
 
 <para>
   Plots in &kst; are created by building up objects into the displayed curves.  In &kst;, there are 5 major classes: 
 </para>
 <itemizedlist>
 <listitem>
 <para>
   Data Sources: data files which are recognized by &kst;.
 </para>
 </listitem>
 <listitem>
 <para>
   Primitives: These are basic data types, including Strings, Scalars (which are single numbers), Vectors (which are ordered lists of numbers) and Matrices (which are 2D arrays of numbers).
 </para>
 </listitem>
 <listitem>
 <para>
   Relations: these objects describe how vectors or matrices are displayed in a plot.  They include Curves (which display an XY pair of vectors) and Images (which display matrices).
 </para>
 </listitem>
 <listitem>
 <para>
   Data Objects: these classes take Primitives as inputs, process them, and output Primitives.  These include Spectra, Histograms, Equations, Fits, Filters, and other Plugins.
 </para>
 </listitem>
 <listitem>
 <para>
   View Items: these are objects that can be drawn, and include plots, labels, lines, etc.  Plots can display Relations (curves and images).  Labels can display Scalars and Strings.
 </para>
 </listitem>
 </itemizedlist>
 
 <para>
   As an example of how these various classes work together, consider the example session in the chapter on 
   <link linkend="tutorial-filters">Filters</link>.  In this session, a curve from a data file was plotted, along with a low pass filtered version of the curve. The resulting data structures are as follows:
 </para>
 
 
 <para>
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Diagram-datatypes-example.png" format="PNG"/>
 </imageobject>
 <textobject>
 <phrase>&kst; Data Types</phrase>
 </textobject>
 </inlinemediaobject>
 </para>
 
 <para>
   The plot displays two curves.  One curve takes two data vectors (INDEX and Column 2) as inputs.  The other takes INDEX as its X vector, and the output vector of the Low Pass Filter as its Y vector.  The low pass filter takes the Column 2 vector, and two Scalars as its inputs.  The two data vectors get their data from the Data Source.
 </para>
 
 <para>
   The data manager for this sessions is shown below. Note that the literal scalars [4 (X30)] and [0.05 (X29)] are not listed.  To keep things clean, and because '4' is not editable, literal scalars like this are not presented in the UI.
 </para>
 
 <screenshot>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-datamanager2.png" format="PNG"/>
 </imageobject>
 </mediaobject>
 </screenshot>
 
 <para>
   This structure could have been chained together further.  For example, the output of the Filter could have been used as the input to a Histogram, and the Histogram of the output of the filtered data could have been plotted instead.
 </para>
 
 <para>
 Descriptions of each data type are provided below.
 </para>
 
 <sect2 id="vectors">
 <title>Vectors</title>
 <para>
 Vectors are ordered lists of numbers.  They are used as the inputs to Data Objects.  They are also used to define the X or Y axis for curves.  While different types of vectors are created in different ways, they can all be used in Data Objects or curves in the same way.
 </para>
 
 <para>
   
 <itemizedlist>
 <listitem>
 <para>
   Data Vectors acquire their data from Data Sources (ie, data files).  They can be created from the <guimenuitem>Vector</guimenuitem> option in the <guimenu>Create</guimenu> menu, or by selecting the
   
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Icon-kst-vectornew.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
 
   icon in any vector selector.  
 </para>
 </listitem>
 <listitem>
 <para>
   Generated Vectors are lists of equally spaced numbers whose range and spacing is defined in the GUI.  They can be created from the <guimenuitem>Vector</guimenuitem> option in the <guimenu>Create</guimenu> menu, or by selecting the
   
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Icon-kst-vectornew.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
 
   icon in any vector selector.  
 </para>
 </listitem>
 <listitem>
 <para>
   Editable Vectors have their data defined through the Python interface.  They can not be created or edited in the GUI.
 </para>
 </listitem>
 <listitem>
 <para>
    Output Vectors are the output of data objects, such as histograms or filters.  
 </para>
 </listitem>
 </itemizedlist>
   
 </para>
 
 </sect2>
 
 <sect2 id="scalars">
   <title>Scalars</title>
   <para>
     Scalars are single numbers that can be used in labels and equations.
   </para>
   
   <itemizedlist>
     <listitem>
       <para>
         Generated Scalars are a single number whose value is defined by the user.  To create a Generated Scalar, select <guilabel>Generate</guilabel> in the <guilabel>New Scalar</guilabel> dialog, opened by selecting <guimenuitem>Scalar</guimenuitem> from the <guimenu>Create</guimenu> menu.
       </para>
     </listitem>
     <listitem>
       <para>
         Data Vector Field based Scalars are generated through Data Sources from the a single specified from a vector field on disk. To create a Data Vector Field based Scalar, select <guilabel>Read from data vector</guilabel> from the <guilabel>New Scalar</guilabel> dialog under <guimenuitem>Scalar</guimenuitem> in the <guimenu>Create</guimenu> menu.  You will specify the data file to read from, the Field name for the vector, and the index of the field you want to read from.  In real time applications, one might want to only read the most recent value.  In this case, select <guilabel>last frame</guilabel> rather than entering a frame number.
       </para>
     </listitem>
     <listitem>
       <para>
         Data Scalars are scalars output directly by data sources.  Most datasources provide <literal>FRAMES</literal> which gives the number of frames in the data file.  Some data sources (eg, dirfiles) also support user defined scalar values.  These are read by selecting <guilabel>Read from data source</guilabel> from the <guilabel>New Scalar</guilabel> dialog under <guimenuitem>Scalar</guimenuitem> in the <guimenu>Create</guimenu> menu. 
       </para>
     </listitem>
     <listitem>
       <para>
         Slave scalars are created automatically by each vector which has been created in &kst;.  Many of these are particularly useful in text labels.  The automatically created slave scalars are:
       </para>
       <para>
         <informaltable>
           <tgroup cols="2">
             <thead>
               <row>
                 <entry>Scalar Name</entry>
                 <entry>Description</entry>
               </row>
             </thead>
             
             <tbody>
               
               
               <row>
                 <entry>NS</entry>
                 <entry>The number of data points in the vector.</entry>
               </row>
               
               <row>
                 <entry>First</entry>
                 <entry>The first data point in the vector.</entry>
               </row>
               
               <row>
                 <entry>Last</entry>
                 <entry>The last data point in the vector.</entry>
               </row>
               
               <row>
                 <entry>Min</entry>
                 <entry>The minimum value in the vector.</entry>
               </row>
               
               <row>
                 <entry>iMin</entry>
                 <entry>the index in the vector of the minimum value in the vector.  The first sample in the vector is index 0.</entry>
               </row>
               
               <row>
                 <entry>Max</entry>
                 <entry>The maximum value in the vector.</entry>
               </row>
               
               <row>
                 <entry>iMax</entry>
                 <entry>the index in the vector of the maximum value in the vector.  The first sample in the vector is index 0.</entry>
               </row>
               
               <row>
                 <entry>Mean</entry>
                 <entry>The mean of the data points in the vector.</entry>
               </row>
               
               <row>
                 <entry>Sigma</entry>
                 <entry>The standard deviation of the data points in the vector.</entry>
               </row>
               
               <row>
                 <entry>Sum</entry>
                 <entry>The sum of the data points in the vector.</entry>
               </row>
               
               <row>
                 <entry>SumSquared</entry>
                 <entry>The sum of the squares of the data points in the vector.</entry>
               </row>
               
               <row>
                 <entry>Rms</entry>
                 <entry>The square root of the mean of the squares of the data points in the vector.</entry>
               </row>
               
               <row>
                 <entry>MinPos</entry>
                 <entry>The smallest positive number in the vector.</entry>
               </row>
               
             </tbody>
           </tgroup>
         </informaltable>
       </para>
       <para>
         The names for Slave Scalars are &lt;Vector Name&gt;:&lt;Scalar Name&gt; (X&lt;N&gt;).  For example, the name of the scalar giving the last sample of the Vector <literal>Az (V2)</literal> would be <literal>Az:Last (X18)</literal>. 
       </para>
     </listitem>
   </itemizedlist>
 </sect2>
 
 <sect2 id="curves">
 <title>Curves</title>
 <para>
 Curves are used to create plottable objects from vectors.  Curves are created from two vectors - an <quote>X axis vector</quote> and a <quote>Y axis vector</quote>.  These two vectors are interpreted as a set of (X,Y) pairs to be plotted.  When the X and Y vectors have the same length, the interpretation is obvious.  
 </para>
 <para>
 If, however, the X vector is of a different length than the Y vector, then the first and last points of each are assumed to represent the first and last (X,Y) pair, and the shorter vector is resampled using linear interpolation to have the same number of samples as the longer vector.
 </para>
 <para>
 Curves are created by the data wizard, from the creation dialog from Data Objects (such as histograms) or by using the <guimenuitem>Curve</guimenuitem> option in the <guimenu>Create</guimenu> menu.  The latter produces the following:
 </para>
 
 <screenshot>
 <screeninfo>New Curve</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-newcurve.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>New Curve</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 <para>
   Here, in the <guilabel>Curve Contents</guilabel> box, the curve has been set up to use INDEX (V1) as the X axis vector and Column 2 (V3) as the Y axis vector.    Note that vectors holding X and Y axis error bars can also be selected.  The 
   
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Icon-kst-vectornew.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
 
   icon in any of the vector selectors will bring up a new vector dialog.  The
   
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Icon-kst-vectoredit.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
   icon will edit the selected vector.
 </para>
 
 <para>
   The appearance of curves is adjusted in the <guilabel>Appearance</guilabel> box.  Some of the flexibility of curves in kst is shown in the next figure. 
   Note that the options are not exclusive - for example, <guilabel>Lines</guilabel> and <guilabel>Points</guilabel> can both be selected.  The <guilabel>Size</guilabel> field specifies the dimensions of display elements such as points and error flags in points (the same way as font sizes are defined.)
   The <guilabel>Weight</guilabel> field specifies the width of lines,
   bar graph borders, and the strokes for points.  The color selector to the right of the example line sets the color of lines, points, and bargraph borders.  The color selector to the right of the <guilabel>Bargraph</guilabel> checkbox sets the fill color for bargraphs.  The last (most recent) point of a
   curve can be indicated by selecting <guilabel>Head</guilabel> and specifying
   a point type and color. The color selector to the right of the <guilabel>Head</guilabel> sets the color for this point.  
 </para>
 
 <screenshot>
 <screeninfo>Curve Demo</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-curvedemo.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>Curve Demo</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 
 <para>
   The <guilabel>Placement</guilabel> box specifies what plot the curve will be displayed in.  Both the <guilabel>Placement</guilabel> and <guilabel>Appearance</guilabel> boxes appear in data object creation dialogs as well, and work the same way.
 </para>
 
 </sect2>
 
 <sect2 id="equations">
 <title>Equations</title>
 <para>
   Equations are data objects whose outputs are: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector (<literal>...:y</literal>) which is the function of one or more data vectors.
     </para>
   </listitem>
   <listitem>
     <para>
       A vector (<literal>...:x</literal>) which passes through the X input vector.  
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   The inputs are: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector which is used as the <literal>x</literal> variable in equations.
     </para>
   </listitem>
   <listitem>
     <para>
       Any vectors or scalars specified by name in the equation text.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   Equations are used to produce vectors which are the point-by-point function of one or more input vectors and scalars.  They are created by selecting <guimenuitem>Equation</guimenuitem> from the <guimenu>Create</guimenu> menu.  An example of creating an equation, and the resulting plot is shown below. In this example, a Generated Vector consisting of 1000 points from -10 to 10 was selected for the x vector.  Recall that a Generated vector can be created by selecting the new vector icon, 
 
   <inlinemediaobject>
   <imageobject>
   <imagedata fileref="Icon-kst-vectornew.png" format="PNG"/>
   </imageobject>
   </inlinemediaobject>
 
   which appears to the right of the <guilabel>X Vector</guilabel> field.  The equation, <literal>sin(x)/x</literal>, was entered into the <guilabel>Equation</guilabel> field.  
 </para>
 
 <screenshot>
 <screeninfo>New Equation</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-newequation.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>New Equation</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 <screenshot>
 <screeninfo>sin(x)/x</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-sinx_x.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>sin(x)/x</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 <para>
   Equations support the following operators:
 </para>  
 
 <itemizedlist>
   <listitem>
     <para>
       Arithmetic operators: <literal>+</literal>, <literal>-</literal>, <literal>*</literal>, <literal>/</literal>, <literal>%</literal> (modulus operator) and <literal>^</literal> (power operator).  
     </para>
   </listitem>
   <listitem>
     <para>
       Bitwise operators: <literal>&amp;</literal>, <literal>|</literal>.  These operators assume the vector is comprised of integers.
     </para>
   </listitem>
   <listitem>
     <para>
       Logical operators: <literal>!</literal>, <literal>&amp;&amp;</literal>, <literal>||</literal>, <literal>&lt;</literal>, <literal>&lt;=</literal>, <literal>==</literal>, <literal>&gt;=</literal>, <literal>&gt;</literal>, and <literal>!=</literal>.  These functions output 1 for True and 0 for False.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   Functions supported by kst are: 
 </para>
 
 <itemizedlist>
   <listitem>
     <para>
       Trig functions working in Radians:  <literal>SIN()</literal>, <literal>COS()</literal>, <literal>TAN()</literal>, <literal>ASIN()</literal>, <literal>ACOS()</literal>, <literal>ATAN()</literal>, <literal>ATAN2()</literal>, <literal>SEC()</literal>, <literal>CSC()</literal> and <literal>COT()</literal>.
     </para>
   </listitem>
   <listitem>
     <para>
       Trig functions working in Degrees:  <literal>SIND()</literal>, <literal>COSD()</literal>, <literal>TAND()</literal>, <literal>ASIND()</literal>, <literal>ACOSD()</literal>, <literal>ATAND()</literal>, <literal>SECD()</literal>, <literal>CSCD()</literal> and <literal>COTD()</literal>.
     </para>
   </listitem>
   <listitem>
     <para>
       Other functions:  <literal>ABS()</literal> (absolute value), <literal>SQRT()</literal> (square root), <literal>CBRT()</literal> (cube root), <literal>SINH()</literal>, <literal>COSH()</literal>, <literal>TANH()</literal>, <literal>EXP()</literal>, <literal>LN()</literal> (natural logarithm), <literal>LOG()</literal> (base 10 logarithm) and <literal>STEP()</literal> (returns 1 if the argument is greater than 0, and 0 otherwise).
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
     Equations also support the constants <literal>PI</literal> and <literal>e</literal>.
 </para>
 
 <para>
   Equations can use any vector or scalar as their input vectors, not just the X vector.  In the next example, the bottom right plot shows the signal in Column 2 with the signal in Column 1 regressed out of it. This has been done by subtracting Column 1, scaled by the slope of a fit to Column 2 vs Column 1, from Column 2.  The fit had been created previously using the <guimenuitem>Fit</guimenuitem> option in the right mouse button menu of the top right plot.
 </para>
 
 <screenshot>
 <screeninfo>Regression</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-eq2Plots.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>Regression</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 <para>
   The <guilabel>New Equation</guilabel> dialog which created this plot is shown below. Note that vectors are identified by enclosing their names in <literal>[  ]</literal>.   So Column 2 is indicated by <literal>[Column 2 (V2)]</literal>.  The <guilabel>Equation</guilabel> line widget has a fairly powerful autocomplete mechanism with a scrollable list of all possible scalars (in its first column) or vectors (in its second column) as you enter the name of the object.  Similarly, the auto complete lists all valid functions and operators as relevant while you type.  The <literal>Esc</literal> key hides the autocomplete widget.
 </para>
 
 <para>
   If the vectors were set to <guilabel>Read to end</guilabel> mode, all elements would be updated real  time as new data came in.
 </para>
 
 <screenshot>
 <screeninfo>Complex Equation</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-eq2.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>Complex Equation</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 </sect2>
 
 <sect2 id="Histograms">
 <title>Histograms</title>
 
 <para>
   Histograms are data objects whose outputs are: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector (<literal>...:num</literal>) which contains the (optionally normalized) count of samples from the input vector which lie within each interval.
     </para>
   </listitem>
   <listitem>
     <para>
       A vector (<literal>...:bin</literal>) which contains the center of each interval for which the counts have been calculated.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   The input is: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector for which the histogram is calculated.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
 In the <guilabel>New Histogram</guilabel> dialog, the bins can be set manually, can be preset once by selecting <guilabel>Auto Bin</guilabel> or can be set to be automatically reset with each data update by selecting <guilabel>Real-time auto bin</guilabel>.
 </para>
 
 <para>
     By selecting <guilabel>Bargraph</guilabel> in the dialog, the histogram can be shown in the standard bar-graph form, below.
 </para>
 
 <screenshot>
 <screeninfo>New Histogram</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-new-histogram.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>New Histogram</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 <screenshot>
 <screeninfo>Histogram</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-histogram.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>Histogram</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 </sect2>
 
 <sect2 id="power-spectra">
 <title>Power Spectra</title>
 
 <para>
   Power Spectra are data objects whose outputs are: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector (<literal>...:psd</literal>) which contains the fft-based spectrum of the input
   vector.
     </para>
   </listitem>
   <listitem>
     <para>
       A vector (<literal>...:f</literal>) which contains the centers of the corresponding
   frequency bins.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   The input is: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector for which the power spectrum is calculated.  Uniform sampling is assumed.
     </para>
   </listitem>
 </itemizedlist>
 
 
 <para>
   The following plot shows an example spectrum.  The plot has been converted to log-log mode (hit 'l' and 'g' in the plot window to toggle Y and X log axes respectively).
 </para>
 
 <screenshot>
 <screeninfo>Spectrum</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-spectrum_log.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>Spectrum</phrase>
 </textobject>
 </mediaobject>
 </screenshot>
 
 
 <para>
   The spectrum dialog (select <guimenuitem>Power Spectrum</guimenuitem> from the <guimenu>Create</guimenu> menu) used to create this plot is shown below:
 </para>
 
 <screenshot>
 <screeninfo>New Spectrum Dialog</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-new-spectrum.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>The New Spectrum Dialog</phrase>
 </textobject>
 </mediaobject>
 </screenshot>  
 
 <para> 
   The dialog entries are as follows:
 </para>
 
 <variablelist>
 <varlistentry>
 <term>
 <guilabel>Data vector</guilabel>
 </term>
 <listitem>
 <para>
 The data vector to create a power spectrum from.
 </para>
 </listitem>
 </varlistentry>
 
 <varlistentry>
 <term>
 <guilabel>Remove Mean</guilabel>
 </term>
 <listitem>
 <para>
 Remove a constant from the input vector to make it mean zero before calculating the spectrum.
 </para>
 </listitem>
 </varlistentry>
 
 <varlistentry>
 <term>
 <guilabel>Apodize</guilabel>
 </term>
 <listitem>
 <para>
 Apodize the data with the selected function before calculating the power spectrum to reduce
 bin to bin leakage.  The default is a Hanning Window.
 </para>
 </listitem>
 </varlistentry>
 
 <varlistentry>
 <term>
 <guilabel>Interleaved average</guilabel> and <guilabel>FFT Length</guilabel>
 </term>
 <listitem>
 <para>
   When <guilabel>Interleaved average</guilabel> is not set, the spectrum is based on an FFT 
   whose length is power of two larger or equal to the length of the input vector.  The remaining points are zero padded.  For cases like this, apodization and mean removal is quite important.
 </para>
 <para>
   When <guilabel>Interleaved average</guilabel> is set, the spectrum is based on the average of FFTs of length <literal>2^x</literal> where <literal>x</literal> is specified by the <guilabel>FFT Length</guilabel> entry, interleaved such that no zero padding is required.  Choosing this option reduces the noise of the spectrum, at the cost of reduced resolution.
 </para>
 </listitem>
 </varlistentry>
 
 <varlistentry>
 <term>
 <guibutton>Sample rate</guibutton>
 </term>
 <listitem>
 <para>
 The frequency bin output vector (<literal>...:f</literal>) will be calculated assuming the input vector was uniformly sampled at this sample rate.</para>
 </listitem>
 </varlistentry>
 
 <varlistentry>
 <term>
 <guilabel>Data units</guilabel> and <guilabel>Rate units</guilabel>
 </term>
 <listitem>
 <para>
 Auto-generating axes labels for plots will be based on these units.
 </para>
 </listitem>
 </varlistentry>
 </variablelist>
 
 </sect2>
 
 <sect2 id="fits">
 <title>Fits</title>
 <para>
   Fits are data objects whose outputs are: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector (<literal>...:Fit</literal>) which is the fit to the data, evaluated at the X value corresponding to each input Y point.
     </para>
   </listitem>
   <listitem>
     <para>
       A vector (<literal>...:Residuals</literal>) which is the difference between the input Y vector and the fit.
     </para>
   </listitem>
   <listitem>
     <para>
         One or more named scalars which correspond to the fit parameters.  For example, for linear fits, the scalars are <literal>...:Intercept</literal> and <literal>...:Gradient</literal>.
     </para>
   </listitem>
   <listitem>
     <para>
         A scalar (<literal>...:chi^2/nu</literal> which holds the reduced chi squared of the fit.
     </para>
   </listitem>
   <listitem>
     <para>
       A vector (<literal>...:Covariance</literal>) which holds the Covariance matrix of the fit in an arbitrary order.  Because the parameters are listed in an arbitrary order, this vector is not currently particularly useful.
     </para>
   </listitem>
   <listitem>
     <para>
       A vector (<literal>...:Parameters</literal>) which is a list of the fit parameters in some arbitrary order.  This vector is rarely useful and may be removed in the future.  The named parameter scalars are a much more useful interface to the fit parameters.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   The inputs are: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector which is to be fit to. (The Y axis vector)
     </para>
   </listitem>
   <listitem>
     <para>
       A vector which is used to generate the X values corresponding to the Y vector.  Note that if the X vector is not the same length as the Y vector, then the X vector will be resampled to have the same number of points as the Y vector in order to generate a series of (X, Y) pairs.
     </para>
   </listitem>
   <listitem>
     <para>
       If a weighted fit is chosen: A vector which describes the error bars for the Y vector.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   A number of fits are available in kst, including weighted (in which the error bar for each Y value is specified) and unweighted fits to lines, polynomials, Gaussians, Lorentians, and exponentials.
 </para>
 
 <para>
   The easiest way to create a fit is by selecting <guimenuitem>Create fit</guimenuitem> from the
   plot context menu (right click in the plot, and then selecting the curve you would like to fit.  The following dialog will appear.
 </para>
 
 <screenshot>
 <screeninfo>New Spectrum Dialog</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-fitdialog.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>The Create Fit Dialog</phrase>
 </textobject>
 </mediaobject>
 </screenshot>  
 
 <para>
   A linear fit has been selected in the <guilabel>Plugin Selection</guilabel> combo box. The X and Y vectors have been automatically selected from the curve which was selected and can not be changed.  The curve properties and placement of the automatically generated curve can be selected as usual.
 </para>
 
 <para>
   When <guibutton>Ok</guibutton> has been selected, the curve is placed in the selected plot, and a label with the fit parameters is automatically created.  Click the mouse wherever you want the label to go.
 </para>
 
 <para>
   You can also create a fit plugin by selecting the appropriate fit from the <guimenu>Fit plugin</guimenu> submenu in the <guimenu>Create</guimenu> menu.  With this dialog you can select the input vectors, but it does not automatically create a curve.  You will have to create a curve by hand by selecting <guimenu>Curve</guimenu> in the <guimenu>Create</guimenu> menu.
 </para>
 
 </sect2>
 
 <sect2 id="filters">
 <title>Filters</title>
 <para>
   Filters are data objects whose output is: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector (<literal>...:Filtered</literal>) which is the same size as the input vector.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   The inputs are: 
 </para>
 <itemizedlist>
   <listitem>
     <para>
       A vector which is to be filtered.
     </para>
   </listitem>
   <listitem>
     <para>
       A number of numbers or scalars which are parameters for the filter.
     </para>
   </listitem>
 </itemizedlist>
 
 <para>
   A number of filters are available in kst.  The band pass, band stop, high pass and low pass filters are conventional zero phase shift Fourier domain filters whose band edges follow the shape of a Butterworth filter.  A higher order filter is a steeper cutoff. 
 </para>
 
 <para>
   Numerical Integrals can be created with the Cumulative Sum filter, and Numerical Derivatives with the Fixed Step Differentiation filter.  In both of these dX takes the size of the step between samples.
 </para>
 
 <para>
   For fields such as angles which have (for example) a discontinuity at 360 degrees, the Unwind Filter can be used to make the signal continuous.  So if the unfiltered signal goes from 359.5 degrees to 0.5 degrees in consecutive samples, the filtered signal will go from 359.5 degrees to 360.5 degrees.
 </para>
 
 <para>
   The Flag filter can be used to mask a vector with NaNs whenever certain bit patters in the flag field are true.
 </para>
 
 <para>
   The easiest way to create a filter is by selecting <guimenuitem>Create filter</guimenuitem> from the
   plot context menu (right click in the plot, and then selecting the curve you would like to fit.
 </para>
 
 <para>
   You can also create a filter plugin by selecting the appropriate filter from the <guimenu>Filter plugin</guimenu> submenu in the <guimenu>Create</guimenu> menu.  With this dialog you can select the input vectors, but it does not automatically create a curve.  You will have to create a curve by hand by selecting <guimenu>Curve</guimenu> in the <guimenu>Create</guimenu> menu.
 </para>
 
 </sect2>
 
 <sect2 id="plugins">
 <title>Standard Plugins</title>
 <para>
   Plugins that do not fit the requirements of being either fits are filters can be created from the <guimenu>Standard Plugin</guimenu> submenu in the <guimenu>Create</guimenu> menu.  They are not well documented.
 </para>
 </sect2>
 
 
 <sect2 id="arrays">
 <title>Matrices</title>
 <para>
 Matrices are two dimensional tables of numbers.  They can be used as the inputs to Data Objects.  They are also used to define the values for Images.  While different types of Matrices are created in different ways, they can all be used in Data Objects or Images in the same way.
 </para>
 
 <para>
   
 <itemizedlist>
 <listitem>
 <para>
   Data Matrices acquire their data from Data Sources (ie, data files).  They can be created from the <guimenuitem>Matrix</guimenuitem> option in the <guimenu>Create</guimenu> menu, or by selecting the
   
 <inlinemediaobject>
 <imageobject>
 <imagedata fileref="Icon-kst-matrixnew.png" format="PNG"/>
 </imageobject>
 </inlinemediaobject>
 
   icon in any matrix selector (for example, in the Image dialog).
 </para>
 </listitem>
 <listitem>
 <para>
   Editable Matrices have their data defined through the Python interface.  They can not be created or edited in the GUI.
 </para>
 </listitem>
 <listitem>
 <para>
    Output Matrices are the output of data objects, such as Spectrograms.  
 </para>
 </listitem>
 </itemizedlist>
 </para>
 
 <screenshot>
 <screeninfo>New Matrix Dialog</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-newmatrix.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>The New Matrix Dialog</phrase>
 </textobject>
 </mediaobject>
 </screenshot>  
 
 <para>
   Matrices can be read from:
 </para>
 
 <para>
 <itemizedlist>
 
   <listitem>
     <para>
       Any data file compatible with QImage - (jpg, png, tiff, bmp, gif, etc).  For color images, four 
       matrices can be read: RED, GREEN, BLUE, and GRAY.
     </para>
   </listitem>
   <listitem>
     <para>
       conventional 2D FITS images (if built with cfitsio).
     </para>
   </listitem>
   <listitem>
     <para>
       BIT Image Streams (BIS) files.
     </para>
   </listitem>
 </itemizedlist>
 </para>
 
 <para>
 The BIS data source can provide matrices from an image stream.  In these cases, the frame number can be selected when the Matrix is created.
 </para>
 
 </sect2>
 
 <sect2 id="images">
 <title>Images</title>
 
 <para>
 Images are used to create plottable objects from Matrices.  The pixels are directly mapped from the matrix.  ie, rows in the matrix are rows in the image.  Columns in the matrix are columns in the image.  The value of the Matrix sets the color of the pixel.
 </para>
 
 <para>
   Images are typically created from selecting <guimenuitem>Image</guimenuitem> from the <guimenu>Create</guimenu> menu.  The Image dialog is shown below:
 </para>
 
 <screenshot>
 <screeninfo>New Image Dialog</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-new-image.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>The New Image Dialog</phrase>
 </textobject>
 </mediaobject>
 </screenshot>  
 
 <para>
   A matrix has been read from a png file, and selected in the Matrix selector (<literal>GRAY (M1)</literal>).  A color map rather than a contour map has been selected, and a grey scale color pallet has been selected.  With <guilabel>Real-time auto threshold</guilabel> selected, the maximum value in the matrix will be set to the maximum value of the color pallet, and the minimum value in the matrix will be set to the minimum value of the color pallet.  All other values will be linearly interpolated.  Alternatively, the maximum and minimum values can be set once, either using <guilabel>Smart</guilabel>/<guilabel>Percentile</guilabel> tool, or by manually setting the thresholds.
 </para>
 
 <para>
   The resulting image is shown below.  Note that, by default, the image will automatically fill the plot window, and will not preserve aspect ratio.
 </para>
 
 <screenshot>
 <screeninfo>New Image Dialog</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-image1.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>The New Image Dialog</phrase>
 </textobject>
 </mediaobject>
 </screenshot>  
 
 <para>
   The aspect ratio can be normalized by selecting <guimenuitem>Normalize X-Axis to Y-Axis</guimenuitem> in the <guimenu>X Zoom</guimenu> submenu of the <guimenu>Zoom</guimenu> plot context menu, or by pressing the "n" key in a plot window.  The image will now have square pixels.
 </para>
 
 <screenshot>
 <screeninfo>New Image Dialog</screeninfo>
 <mediaobject>
 <imageobject>
 <imagedata fileref="Screenshot-kst-image2.png" format="PNG" />
 </imageobject>
 <textobject>
 <phrase>The New Image Dialog</phrase>
 </textobject>
 </mediaobject>
 </screenshot>  
 
 <para>
   The range of the color pallet can be adjusted from the curve edit dialog, or by pressing 'i' in an image.  This will cycle the color limits, allowing an increasing fraction of the pixels to be saturated at either end of the color scale before returning to full range.
 </para>
 
 </sect2>
 
 </sect1>
 </chapter>
 
 <!-- Keep this comment at the end of the file
 Local variables:
 mode: xml
 sgml-omittag:nil
 sgml-shorttag:nil
 sgml-namecase-general:nil
 sgml-general-insert-case:lower
 sgml-minimize-attributes:nil
 sgml-always-quote-attributes:t
 sgml-indent-step:0
 sgml-indent-data:true
 sgml-parent-document:("index.docbook" "book" "chapter")
 sgml-exposed-tags:nil
 sgml-local-catalogs:nil
 sgml-local-ecat-files:nil
 End:
 -->