Warning, /graphics/kst-plot/devel-docs/fits_datasource.txt is written in an unsupported language. File is not indexed.

 FITS Datasource Design 
 ============================
 Revision History
 ----------------------------
 
 2006 05 29 : Ted K.  More details about FITS helper library
 2006 05 04 : Ted K.  Added info about WCS
 2006 05 03 : Ted K.  Initial proposal
 ----------------------------
 
 This is a proposal for a new general FITS datasource (called fitsfile).  There
 are some auxillary issues that also need to be dealt with in the healpix and
 LFIIO datasources.
 
 
 Utility library
 --------------------------------------------------------------------
 
 The cfitsio library has low level functions to access the FITS file.  It
 would be convenient to have a library of higher level functions that could be
 called from all datasources that deal with FITS files.  Examples of
 possible functions would be:  fetch all keys, build combined fieldlist from
 all extensions, build matrixlist from all extensions, etc.
 
 In order to be really useful, this utility library needs to be accessible from
 the understands_*, fieldList_* and matrixList_* functions.  I'm not sure of
 the best place to put such a library in the buildsystem- maybe
 kst/src/datasources/libfitsfile/ ?
 
 Proposed Functions:
 
 1.  fitsNHDU( fitsfile *fp )
     Returns the number of HDU's in the FITS file.
     
 2.  fitsDim( fitsfile *fp, int HDU )
     Returns the dimensions of the image or table contained in the 
     current HDU.  In the case of binary tables with "wide" columns (each
     column entry is a vector), the column entries will be "unrolled" 
     and the returned number of rows will reflect this new size.
     
 3.  fitsKeys( fitsfile *fp, int HDU )
     Returns a list of FITS keys.  This list will contain any keywords in the
     file header, and then the keys contained in the specified HDU.
     
 4.  fitsNames( fitsfile *fp, int HDU )
     Returns a QStringList of the names of all columns in the HDU (if the HDU
     contains a table), or the label of the image.
     
 5.  fitsUnits( fitsfile *fp, int HDU )
     Returns a QStringList of the units of all columns in the HDU (if the HDU
     contains a table), or the units of the image.
     
 6.  fitsFields( fitsfile *fp, int HDU )
     Returns a fieldlist for the current HDU.  Some datasources may wish to
     construct their own fieldlist using the information returned from
     fitsNames and fitsUnits.
     
 7.  fitsMatrices( fitsfile *fp, int HDU )
     Returns a matrixlist for the current HDU.  Some datasources may wish to
     construct their own fieldlist using the information returned from
     fitsNames and fitsUnits.
 
 
 Overview of the fitsfile datasource
 --------------------------------------------------------------------
 
 The overall behaviour of the datasource will be to offer up all data in the
 FITS file as both vectors and matrices.  The datasource will scan through
 the current extension (HDU) in the FITS file and build up a list of 
 available fields.  The current HDU will default to the first one, and be 
 selectable from the config dialog.  The INDEX field length will be determined
 based on the size of the current HDU.
 
 A.  Vector List.  The datasource would call a function in libfitsfile to build
 the fieldlist for the current HDU.  For an image extension, it will read the
 units and add 2 fields to the fieldlist that represent the pixel index (row
 major) and the flat-packed pixel values from the image.  For a table
 extension, it will read the names and units of the columns and combine them
 to create a field name to add to the fieldlist.
 
 B.  Matrix List.  The datasource would call a function in libfitsfile to build
 the matrixlist for the current HDU.  For an image extension, it will read the
 units and add a field to the matrixlist which represents the rectangular 
 image data.  For a table extension, it will read the names and units of all 
 columns and combine them to create a field name to add to the matrixlist.  So 
 for a table extension, the corresponding matrix field has the same dimensions 
 as the table.
 
 C.  Dealing with corrupt or "poorly created" files.  Unfortunately, not all
 FITS files "in the wild" are of the same quality.  Sometimes files become
 truncated or the software creating the file has made a mistake.  In my
 experience, a more robust way of handling these issues is to use the very low
 level functions of cfitsio to read the file.  This makes it easier to handle
 errors with a subset of the data in the file, while keeping any data that is
 good. The higher level functions tend to just give up if they encounter a
 problem.
 
 D.  Configuration.  The initial config widget will be very simple, and allow
 the user to choose which HDU to use in the file.
 
 E.  Command line.  The command line switches -x, -y, and -z will determine how
 the fitsfile datasource returns data to be plotted.  If the -z option is not
 specified, all further actions pertain to the first HDU.  If the -z option is 
 specified alone, the value will be taken as the HDU to plot as an image.  If
 the -x and -y options are specified, then their values correspond to the
 columns of the fieldlist to use.  If no -x is specified, data is plotted
 versus the INDEX field for the selected HDU.
 
 
 WCS Extensions
 --------------------------------------------------------------------
 The WCS system provides a way of specifying the coordinates of each pixel
 value in a FITS image.  This can be done by specifying mapping parameters, or
 even by specifying every pixel's coordinates in a binary table extension. 
 This is great, but what can we do with this information?  The best we can do
 at this point is maybe support some limited types of coordinate mappings
 (rectangular ones).  Full support of WCS projections will not be possible
 with kst until we have a complete 3D display framework.
 
 Note that WCS is implemented as an external library.  It is small and GPL'd,
 so it may be easier to just include the files we wish within the fitsfile
 datasource.  This can be discussed more later.
 
 
 Healpix, LFIIO, and Miscellaneous
 --------------------------------------------------------------------
 
 A.  The healpix datasource needs to support the use case of reading the pixel
 values straight out of the binary table as vectors.  I will do this by adding
 some additional fields to the fieldlist that have the form "(RAW) <field
 name>", so that the user knows what they are getting.
 
 B.  The LFIIO datasource needs to support interleaved data in tables.  My hope
 is that it can make use of some of the existing utility functions in
 libfitsfile. For example, it could call the function to read all keys and then
 check if there are special keys to indicate that the file is an LFIIO fits
 file.  Then it could call another function to build the fieldlist and simply
 add/remove a couple fields to deal with the interleaving.  So basically LFIIO
 would support all the functionality of the fitsfile datasource (through the
 use of libfitsfile), with some "extra" functionality built on top.
 
 C.  Notes:  Need to properly handle null pixel values in images.