next_inactive up previous


VISTA DATA FLOW SYSTEM
(VDFS)
---------------
for VISTA & WFCAM data









WSA Interface Control Document

author
N.C.Hambly (WFAU Edinburgh)
Science Archive Project Scientist
number
VDF-WFA-WSA-004
issue
Issue 2.0
date
July 2003
co-authors
M.J.Irwin & J.R.Lewis, CASU


Contents


SCOPE

This Interface Control Document (ICD) for the WFCAM Science Archive (WSA) describes the data flow subsystem interface between the data processing centre (CASU at the IoA, Cambridge) and the archive centre (WFAU at the IfA, Edinburgh). Details of the types and specifications of processed WFCAM data to be transfered, along with the transfer protocols (file naming, transfer method and procedure), are given. The details of this ICD have been agreed between CASU and WFAU; the formalities are being overseen by the JAC and the work is part of the VISTA Data Flow System (VDFS) project.

The ICD is intended to be a formal interface control agreement between the WFCAM data processing centre at CASU and the archive centre at WFAU in Edinburgh. The processing centre/archive centre interface is the final subsystem interface in the WFCAM data flow chain, and is subject to the rules laid out herein. The ICD concerns WFCAM data only; all other data ingested into the WFCAM Science Archive (WSA) are outside the scope of interface control (the WSA will also ingest publicly released data products, eg. SDSS and 2MASS etc., from other non-CASU sources).

The ICD is meant to be a technical reference: its intended audience is software engineers and scientists working on processing and archiving in the data flow. It takes the form of a formal agreement between CASU and WFAU, but must also satisfy other external bodies, namely JAC and the UKIDSS survey science consortium.


OVERVIEW

This document is structured as follows. In Section 3, we describe the fundamental rules that the interface will adhere to, including a statement of the primary data format, FITS. Then, in Section 4, we describe the top-level specifications for data that will be transfered between Cambridge and Edinburgh, including a description of FITS conventions, keywords, file naming conventions, units, systems of physical quantities, and unified column descriptors. Section 5 goes on to describe in explicit detail the data structures that will be transfered. Then, Section 6 describes the transfer methods and procedures that will achieve the data flow from Cambridge to Edinburgh. Finally, security issues are dealt with in Section 7.

Generally, this document is modelled on the ESO Data Interface Control Document [1], and with the exception of the ESO hierarchical FITS keyword definition, follows as closely as possible the specifications provided therein. A WSA data flow system overview is provided in [2]. Fundamental `meta' data description (ie. FITS frame headers and keywords) are described in [3]. The JAC/CASU interface is defined and described in [4]; CASU pipeline processing is described in [5].

Applicable WSA documents are listed in Section 10.


FUNDAMENTALS

WFAU Ingest

The WSA at WFAU will ingest WFCAM data from CASU; there will be no transfer of WFCAM data between JAC and WFAU for example.

Data transfer method

The WSA will ingest data via the internet; tapes and/or `pluggable' disks will not be employed. The implications for required network bandwidth are discussed in AD01. More details are given in Section 6.

Format

Data output from CASU will be provided in standard FITS format (as specified in [6]) only. Data will not be expressed in any `hierarchical' system, eg. ESO hierarchical FITS, or the UK Starlink Hierarchical Data Structure format (NDFs). The FITS standard is mature, universally accepted and ideal for transporting both bulk pixel and catalogue data. CSAU and WFAU will both use the CFITSIO library [7] to read and write FITS files.

Transfered data

Data transfered from CASU will consist of processed pixels (where the processing steps are specified by the observing protocol used), confidence maps, derived source catalogues and associated description data; no raw pixel data will be transfered to (or held in) the WSA. Where irreversible stages such as stacking or mosaicing have been done as part of the reduction procedure, the individual component images and catalogues will also be transferred. Library calibration frames will also be transfered into the WSA (eg. dark frames, flat fields, master skies) for use by users (not for any processing at the archive end).

Note that for large scale survey programmes where total pixel data volumes are high (eg. the UKIDSS wide area LAS, GPS and GCS - see AD04) the WSA cannot store both individual superframes and contiguous mosaic tiles consisting of $2\times2$ of these non-contiguous units. In this case, only the individual superframe units will be transfered and archived.


TOP-LEVEL DATA SPECIFICATION

Preliminaries

Processed frames will be stored in FITS format, following the guidelines set out in [1]:

Derived source catalogues corresponding to each image extension will be written as FITS binary tables in extensions of a single, separate MEF file with a similarly empty primary array. The headers for the catalogue MEF will contain all the information of image MEF headers plus ancillary processing keywords and values.

General FITS keywords

Keywords will follow the standards set out in [1] and [6] as described (for WFCAM data) in [3]. All keywords and associated values written to the HDS container files produced by the WFCAM DAS must be propagated through the JAC/CASU interface, through the data processing pipeline and into the WSA.

The first keyword in any extension HDU must be XTENSION, and it's value will take on only 'IMAGE ' or 'BINTABLE'; the EXTNAME keyword will be used to identify the extension with a particular device detector and a unique ID for each device used in WFCAM must be propagated through the data flow via an assigned keyword. Binary tables will have every column described by keywords TTYPEn, TFORMn and TUNITn (see later).

World Co-ordinate System (WCS; ie. astrometric) information will be propagated using a set of standard keywords described in the latest FITS WCS proposals [11,12] by Greisen and Calabretta.

Physical units

Physical units will comply with SI units and their derivatives with a few exceptions for astronomical convenience (see [1] Section 9, Table 14).

Celestial co-ordinates will be expressed in a time system described by primary HDU keyword RADECSYS; it is anticipated that this will have value 'FK5' (ie. Hipparcos/Tycho ICRS) over the lifetime of WFCAM, but this may of course change for VISTA.

File naming conventions

CASU/JAC/ATC have an agreed policy on filenames; furthermore, it is UKIRT policy to use run numbers that reset back to 1 each night. For ease of tracking files through the data flow system, the CASU/WFAU interface will follow the same policy, with conventions for processed products, as follows.

At the telescope, the DAS will produce files called wayyyymmdd_12345.sdf, wbyyyymmdd_12345.sdf and so on, where the a,b,c,d correspond to detector, w stands for wfcam and 12345 is the 5 digit run number.

CASU will create 2D raw MEF files from the individual NDFs as a precursor to input to the processing pipeline front-end, with names of the form w_yyyymmdd_nnnnn.fit and processed filenames of the form w_yyyymmdd_nnnnn_suffix.fit where yyyymmdd is the UT date of observation, nnnnn is the UKIRT DAS running number (reset to 1 on a nightly basis) and _suffix is a combination of an underscore character plus two-letter abbreviations indicating pipeline processing actions: _sf = interleaved superframe, _st = stack, _sf_st = stacked superframe, _sf_tl = tiled superframe etc. Catalogues generated from frames will be rootname_cat.fits and confidence maps for frames rootname_conf.fit, etc.

When a file is the result of a combination of several files, the run number of the first file in the list of combined files will be used for the filename of the combined data file.


DETAILED DATA SPECIFICATION

Data obtained at the time of observation

Observations will be described via the keywords OBSERVER, USERID, OBSREF, PROJECT, MSBID and OBJECT keywords.

Note that the OBJECT keyword MUST be a field identifier for survey observations for the purposes of curation within the archive.

Instrumental characteristics, set-ups and parameters will be described by keywords as detailed in [3], including instrument detector configuration (eg. array used DETECTOR; number of integrations NINT), detector controller information (eg. camera read mode READMODE; read-out application CAPPLICN), optical configuration (eg. filter name FILTER; base focus position FOC_MM) and observing conditions/environment (eg. air temperature AIRTEMP; relative humidity HUMIDITY; opacity data CSOTAU).

All these FITS keys will be propagated through the data flow chain from the DAS to the WSA.

Data products (ie. derived data)

Corrected pixel data

The CASU pipeline will instrumentally correct WFCAM pixels into a product that is instrument-signature free. The reduction steps involved in doing so, the derived astrometric and (first-cut) photometric calibrations and resulting DQC information generated will be propagated into the WSA using FITS keys detailed in the appendices in Section 8. Appendix 8.1 shows example FITS keys for the primary HDU; Appendix 8.2 shows an example of an extension set. These represent the current status of the header definitions; discussions are still underway as to propagation of provenance information through the JAC/CASU interface and subsequently through the CASU/WFAU interface.

Library calibration frames will have identical FITS keys to science frames, but library frame keywords for library frames will not refer to other frames (eg. library flatfields will not be flatfielded, etc).

Differences in the FITS keys in primary extension HDUs for combined frame products will be limited to the propagation of provenance information, ie. a list of the individual frames that have been combined in the pipeline to create a combined frame product will be listed as a set of COMMENT keywords.

Pixel data values will be represented in 4-byte integer numbers (ie. BITPIX=+32) and CFITSIO `RICE' tile compression will be employed to facilitate efficient storage and network transfer. Whenever possible, all processing will maintain the original units, ie. if the original raw data run from 0 to 100,000 ADU, the range in data numbers in processed frames will be similar. At this stage, we allow for the posisbility of use of BSCALE and BZERO FITS keywords and values to recast 4-byte integers into floating point numbers.

Source catalogues

The standard set of CASU source detection parameters can be found in [5]. An example FITS header for a catalogue MEF is given in Appendix 8.3. The following are an extract of the corresponding FITS binary table details for each catalogue attribute (TFORM is 1E throughout):

No.  Name                   TTYPE                 TUNIT

 1   Seq. no.               Sequence_number         -
 2   Isophotal flux         Isophotal_flux         ADU
 3   X co-ordinate          X_coordinate          pixels
 4   Error in X             X_coordinate_error    pixels
 5   Y co-ordinate          Y_coordinate          pixels
 6   Error in Y             Y_coordinate_error    pixels
 7   Gaussian sigma         Gaussian_sigma        pixels
 8   Ellipticity            Ellipticity              -
 9   Position angle         Position_angle       degrees
10   Areal profile  1       Areal_1_profile       pixels
.
.
17   Areal profile  8       Areal_8_profile       pixels
18   Peak height            Peak_height            ADU
19   Peak height error      Peak_height_error      ADU
20   Aperture flux 1        Aperture_flux_1        ADU
21   Aperture flux 1 error  Aperture_flux_1_error  ADU
22   Aperture flux 2        Aperture_flux_2        ADU
23   Aperture flux 2 error  Aperture_flux_2_error  ADU
.
.
44   Aperture flux 13       Aperture_flux_13       ADU
45   Aperture flux 13 error Aperture_flux_13_error ADU
46   Petrosian radius       Petrosian_radius      pixels
47   Kron radius            Kron_radius           pixels
48   FWHM radius            FWHM_radius           pixels
49   Petrosian flux         Petrosian_flux         ADU
50   Petrosian flux error   Petrosian_flux_error   ADU
51   Kron flux              Kron_flux              ADU
52   Kron flux error        Kron_flux_error        ADU
53   FWHM flux              FWHM_flux              ADU
54   FWHM flux error        FWHM_flux_error        ADU
55   Error bit flag         Error_bit_flag        flag
56   Sky level              Sky_level              ADU
57   Sky variance           Sky_variance           ADU
58   Child/parent           Parent_or_child_flag  flag
59   Right Ascension        RA                   radians
60   Declination            DEC                  radians
61   Classification         Classification        flag
62   Profile statistic      Class_statistic      N-sigma
63   PSF flux               PSF_flux               ADU
64   PSF flux error         PSF_flux_error         ADU
65   PSF fitted X           PSF_fit_X             pixels
66   PSF fitted X error     PSF_fit_X_error       pixels
67   PSF fitted Y           PSF_fit_Y             pixels
68   PSF fitted Y error     PSF_fit_y_error       pixels
69   PSF fit chi-squared    PSF_fit_chi2            -
70   nu                     PSF_fit_dof             -
71   1D Sersic flux         1D_Sersic_flux         ADU
72   Scale length           1D_Sersic_scale_len     -
73   Power law index        1D_Sersic_index         -
74   Error in 1D fit        1D_Sersic_fit_chi2      -
75   1D Sersic fit nu       1D_Sersic_fit_nu        -
76   2D Sersic flux         2D_Sersic_flux         ADU
77   Scale length           2D_Sersic_scale_len     -
78   Power law index        2D_Sersic_index         -
79   Error in 2D fit        2D_Sersic_fit_chi2      -
80   2D Sersic fit nu       2D_Sersic_fit_nu        -

The attribute set for CASU standard list-driven photometry source remeasurement will similarly be specified by 1/10/2003, at which point WFAU will take delivery of the software to implement this feature within the archive (for more details, see the database design document [10]). This is not strictly speaking an interface control issue (since WFAU will not ingest list-driven catalogue products from the CASU standard pipeline) but is mentioned here for completeness.

Convention for null values

The ANSI/IEEE-754 floating-point number standard defines certain special values that are used to represent such quantities as not-a-number (NaN), denormalized, underflow, overflow, and infinity (see the Appendix in the NOST FITS standard [6] or the NOST FITS User's Guide for a list of these values). The CFITSIO routines that read floating point data in FITS files recognize these IEEE special values and by default interpret the overflow and infinity values as being equivalent to a NaN, and convert the underflow and denormalized values into zeros. In cases where programmers may want access to the raw IEEE values without any modification by CFITSIO, this can be done by calling the fits_read_img or fits_read_col routines while specifying 0.0 as the value of the NULLVAL parameter. This will force CFITSIO to simply pass the IEEE values through to the application program without any modification.

Since most of the binary tables contain floating point numbers there is no need to specify null values as these can be specified transparently in cfitsio. Null floats will be set to FLOATNULLVAL (equivalent to IEEE not-a-number) and the CFITSIO routines used as normal. For any integer columns, the FITS null value will be explicitly defined by the TNULLn keyword.


TRANSFER METHODS & PROCEDURES

Method

Transfer will be via the internet using standard methods. The data to be transferred will reside in Cambridge on specific RAID arrays attached to a linux PC cluster. WFAU will have an account on this system. Directories of processed nights data will be setup as the pipeline is running. While processing and writing to a given directory is still running a directory lock file will be used to denote the `in progress' operations. After completion the lock file will be unset/removed enabling a remote client task to automatically initiate data transfer to Edinburgh. In AD01 we give details of on-going network bandwidth experiments with large data volumes and employing various transfer protocols; transfer methods being tested include scp, GridFTP and sftp (standard ftp will not be used because of it's inherent insecurities). We anticipate that GridFTP will be employed to transfer WFCAM data from CASU to WFAU.

Procedure

Location of data is guaranteed by the pipeline and will be in an observation-date-driven directory structure to which WFAU will have secure, direct access. `Handshaking', eg. notification of readiness, will be achieved using a lockfile system as outlined above; verification of successful transfer will include automatic checking within the transfer utility employed via the number and size of files transferred. Checksum verification will be employed using standard cfitsio routines.

In AD02, we give more details of the transfer task software, including error handling and transfer logging.

Updates

In the event of pipeline reruns over previous data (eg. because of improvements in instrumental correction and/or source extraction software) the interface as a whole will be the same regardless of whether data being transfered is first-run or re-run as long as the archive system can cope with overwriting issues within and storage of repeat data. These are dealt with in the database design presented in AD02.


SECURITY ISSUES

Nightly processed data will be held online at CASU until transfer of those data to WFAU is verified. As noted before, secure transfer protocols will be employed between CASU and WFAU to protect data from malicious corruption or access by unauthorised users. Although not strictly speaking a CASU/WFAU interface issue, raw data will be held in Cambridge as the primary UK backup in case of any catastrophic data loss further down the data flow (raw data will of course also be archived offline at JAC).

Bibliography

1
ESO Data Interface Control Document, GEN-SPE-ESO-19940-794/2.0; available from http://archive.eso.org/DICB/dic-2.0/dic-2.0.4.pdf

2
WFCAM Science Archive overview document;
http://www.roe.ac.uk/~nch/wfcam/VDF-WFA-WSA-001-I1/VDF-WFA-WSA-001-I1.html

3
ATC WFCAM HDS container and FITS headers, WFCAM project Document No. ??, http://www.jach.hawaii.edu/JACpublic/UKIRT/instruments/wfcam/ICD/headers_0p5.html

4
JAC-CASU Interface Control Document,
http://www.jach.hawaii.edu/JACpublic/UKIRT/instruments/wfcam/ICD/

5
CASU WFCAM/VISTA Documentation: Current iteration of agreed parameters for standard catalogue products, http://www.ast.cam.ac.uk/~wfcam/documentation.html

6
Definition of the Flexible Image Transport System (FITS), document NOST 100-2.0 http://fits.gsfc.nasa.gov/fits_home.html

7
CFITSIO - A FITS File Subroutine Library,
http://heasarc.gsfc.nasa.gov/docs/software/fitsio/fitsio.html

8
WFCAM/VISTA Science Archive Development, http://www.roe.ac.uk/~nch/wfcam/

9
WFCAM Science Archive hardware/OS/DBMS design document;
http://www.roe.ac.uk/~nch/wfcam/VDF-WFA-WSA-006-I1/VDF-WFA-WSA-006-I1.html

10
WFCAM Science Archive database design document;
http://www.roe.ac.uk/~nch/wfcam/VDF-WFA-WSA-007-I1/VDF-WFA-WSA-007-I1.html

11
Representations of world co-ordinates in FITS, Greisen E.W., Calabretta M.R., A&A, 395, 1061 (2002)

12
Representations of celestial co-ordinates in FITS, Calabretta M.R., Greisen E.W., A&A, 395, 1077 (2002)

13
The UKIDSS Proposal; http://www.ukidss.org/sciencecase/sciencecase.html


APPENDICES


Primary HDU FITS keys from CASU pipeline-processed image data

SIMPLE  =                    T / file does conform to FITS standard
BITPIX  =                    8 / number of bits per data pixel
NAXIS   =                    0 / number of data axes
EXTEND  =                    T / FITS dataset may contain extensions
COMMENT   FITS (Flexible Image Transport System) format is defined in Astronomy
COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H
INHERIT =                    T
TELESCOP= 'UKIRT             ' / Telescope name
INSTRUME= 'WFCAM             ' / Instrument
DHSVER  = 'UKDHS 2002 Oct 31 ' / Data handling version
HDTFILE = 'wfcam.hdt         ' / Name of hdt file
OBSERVER= 'Daffy Duck        ' / Observers names
USERID  = 'DD      '           / Userid logged in as
OBSREF  = 'notPATT99'          / PATT or other reference
PROJECT = 'Example WFCAM data' / Time-allocation code
MSBID   = 'b44d9b4e3b90e6f99b7c3a032301600b' / Id min.-schedulable block
OBJECT  = 'CIRSI_NGC135A'      / Object name from telescope
RECIPE  = 'SOME_WFCAM_RECIPE'  / Data reduction recipe to be used
OBSTYPE = 'OBJECT  '           / Type (BIAS|DARK|ARC|FLAT|OBJECT|SKY)
OBSNUM  =                    9 / Observation number
GRPNUM  =                    8 / Group number applied to all members
GRPMEM  =                    T / Group membership
TILENUM =                 6523 / Tile number applied to all members
STANDARD=                    F / Is the target a standard star observation?
NOFFSETS=                    4 / Number of offset positions in a pattern
NJITTER =                    4 / Number of positions in tel pattern
JITTER_I=                    1 / Serial number in this tel jitter pattern
JITTER_X=                 0.00 / [arcsec] X (RA) offset in tel jitter pattern
JITTER_Y=                 0.00 / [arcsec] Y (Dec) offset in tel jitter pattern
NUSTEP  =                    4 / Number of positions in microstep pattern
USTEP_I =                    1 / Serial number in this microstep pattern
USTEP_X =                 0.00 / [arcsec] X (RA) offset in microstep pattern
USTEP_Y =                 0.00 / [arcsec] Y (Dec) offset in microstep pattern
NFOC    =                    0 / Number of positions in focus scan
NFOCSCAN=                    0 / Number of focus scans in focus test
UTDATE  = '20010607'           / UT date as integer in yyyymmdd format
DATE-OBS= '2001-06-07T21:23:46Z' / Date and time (UTC) of start of observation
DATE-END= '2001-06-07T21:24:10Z' / Date and time (UTC) of end of observation
WCSAXES =                    2 / Number of axes in world co-ordinate system
RADECSYS= 'FK5     '           / Mean IAU 1984 equatorial co-ordinates
EQUINOX =             2000.000 / [yr] Equinox of object position
RABASE  =             217.4049 / [h] Right ascension of base position
DECBASE =           0.06174444 / [deg] Declination of base position
TRAOFF  =                0.000 / [arcsec] Right ascension telescope offset
TDECOFF =                0.000 / [arcsec] Declination telescope offset
AMSTART =                1.312 / Airmass at start of observation
AMEND   =                1.310 / Airmass at end of observation
TELRA   =             217.4049 / [h] Current telescope right ascension
TELDEC  =           0.06174444 / [deg] Current telescope declination
GSRA    =             217.4049 / [h] Right ascension of guide star
GSDEC   =           0.06174444 / [deg] Declination of guide star
READMODE= 'CDS_v1  '           / Name of camera readmode
CAPPLICN= 'dunno   '           / Name of camera readout application
READOUT = 'CDS     '           / Camera readout (CDS|NDR|SAR|RRR)
EXP_TIME=                24.08 / [s] Integration time per exposure
NEXP    =                    1 / Number of exposures in integration
READINT =             0.300000 / [s] Interval between reads
NREADS  =                    0 / Number of reads per exposure
FILTER  = 'J       '           / Combined filter name
FOC_MM  =               2.4992 / [mm] Focus position
AIRTEMP =               -0.013 / [degC] Air temperature
BARPRESS=              650.000 / Ambient pressure
DEWPOINT=                2.000 / [degC] Dewpoint
DOMETEMP=                1.101 / [degC] Dome temperature
HUMIDITY=               45.816 / Relative Humidity
MIRRBSW =                7.123 / [degC] Temperature mirror B SW
MIRRNE  =                7.124 / [degC] Mirror temperature NE
MIRRNW  =                7.124 / [degC] Mirror temperature NW
MIRRSE  =                7.124 / [degC] Mirror temperature SE
MIRRSW  =                7.124 / [degC] Mirror temperature SW
MIRRBTNW=                7.128 / [degC] Mirror bottom temp. NW
MIRRTPNW=                7.128 / [degC] Mirror top temp. NW
SECONDAR=                7.133 / [degC] Temperature of secondary
TOPAIRNW=                7.134 / [degC] Top air NW
TRUSSENE=                3.286 / [degC] Truss leg ENE
TRUSSWSW=                2.048 / [degC] Truss leg WSW
WINDDIR =              265.958 / [deg] Wind direction, azimuth
WINDSPD =               48.915 / [km/h] Wind speed
CSOTAU  =                0.047 / Tau at 225 GHz from CSO
TAUDATE = '2001-11-30T04:07'   / Time and date opf Tau reading
TAUSRC  = 'CSO     '           /Source of opacity data
CNFINDEX=                    1 / Configuration index
END


Extension HDU FITS keys from CASU pipeline-processed image data

XTENSION= 'IMAGE   '           / IMAGE extension
EXTNAME = '                   '/ Extension name/detector name...?
BITPIX  =                  +32 / number of bits per data pixel
BSCALE  =                  1.0 / 
BZERO   =                  0.0 /
CAMNUM  =                    1 / Number of WFCAM array
HDTFILE2= 'wfcam_w.hdt       ' / Name of camera specific hdt file
NAXIS   =                    2 / number of data axes
NAXIS1  =                 1091 / length of data axis 1
NAXIS2  =                 1091 / length of data axis 2
PCOUNT  =                    0 / required keyword; must = 0
GCOUNT  =                    1 / required keyword; must = 1
DETECTOR= 'ALADDIN'            / Detector array used
NINT    =                    1 / Number of integrations in observation
DROWS   =                 1024 / [pixel] Number of detector rows
DCOLUMNS=                 1024 / [pixel] Number of detector columns
RDOUT_X1=                    1 / Start column of array readout
RDOUT_X2=                 1024 / Start column of array readout
RDOUT_Y1=                    1 / Start row    of array readout
RDOUT_Y2=                 1024 / Start row    of array readout
PIXLSIZE=                0.454 / [arcsec] Pixel size
FLATCOR = 'Done with: J_flat_1.fit'
RSTANOM = 'Done with medlinfilt: 50 25'
CIR_CPM = 'wfcam_6523_conf.fit[1]' / Confidence map
SKYSUB  = 'Done TILE_SKY: sky_6523.fits[0] 1.104' / Sky subtraction info
PROJP1  =                   1.
PROJP3  =                 220.
CIRMED  =              1881.53 / Latest estimate of background
CIR_BVAR=             182.6193 / Latest estimate of background variance
CIR_ZERO=            -73.88457 / Pedestal value relative to group average
CIR_SCAL=                   1. / Background scale relative to group maximum
CIR_OPM = 'irx_6523_c1_012_opm.fits[0]' / Object mask
CTYPE1  = 'RA---ZPN'
CTYPE2  = 'DEC--ZPN'
CRVAL1  =     217.409246142543
CRVAL2  =   0.0631388007913921
CRPIX1  =     1569.17568234834 / Dither offset Y
CRPIX2  =    -402.446920842832 / Dither offset Y
CD1_1   = -1.66959860113266E-06
CD2_1   = 0.000126195921150401
CD1_2   = 0.000126124625720947
CD2_2   =  1.4709458469656E-06
PV2_0   =                   0. /
PV2_1   =                   1. /
PV2_2   =                   0. /
PV2_3   =                 220. /
WCSPASS =                    1 / Pass level of WCS
CIR_XOFF=             35.40594 / Dither offset X
CIR_YOFF=              33.1044 / Dither offset Y
NUMBRMS =                   30
STDCRMS =     0.32526138905959
HISTORY 20030625 15:11:45                                                      
HISTORY    $Id: cir_stage1.c,v 1.5 2003/06/17 09:49:08 jim Exp $               
HISTORY 20030625 15:12:05                                                      
HISTORY    $Id: cir_arith.c,v 1.3 2003/02/03 09:32:36 jim Exp $                
HISTORY 20030625 15:12:21                                                      
HISTORY    $Id: cir_apm.c,v 1.5 2003/02/03 09:32:36 jim Exp $                 
HISTORY 20030625 15:12:46                                                      
HISTORY    $Id: cir_tartup.c,v 1.5 2003/02/03 09:32:36 jim Exp $               
HISTORY 20030625 15:16:32                                                      
HISTORY    $Id: cir_apm.c,v 1.5 2003/02/03 09:32:36 jim Exp $                  
HISTORY 20030625 15:17:05                                                      
HISTORY    $Id: cir_wcsoffset.c,v 1.3 2003/02/03 09:32:36 jim Exp $   
END

BSCALE and BZERO will default to 1.0 and 0.0 respectively if absent from the keyword list. FLATCOR is used to tell the pipeline how to do the flat fielding. If it's been done, then it has the words 'Done with' and the name of the flat field file. RSTANOM is the reset anomaly correction. This shows that it's been done with a median-linear filter with box sizes of 50 and 25 pixels respectively. CIR_CPM is the confidence map.


Example FITS keys from a catalogue MEF

Primary HDU (excluding keys inherited from corresponding image data):

SIMPLE  =                    T / file does conform to FITS standard
BITPIX  =                    8 / number of bits per data pixel
NAXIS   =                    0 / number of data axes
EXTEND  =                    T / FITS dataset may contain extensions
COMMENT   FITS (Flexible Image Transport System) format is defined in Astronomy
COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H
END

Each extension HDU:

XTENSION= 'BINTABLE'           / binary table extension
BITPIX  =                    8 / 8-bit bytes
NAXIS   =                    2 / 2-dimensional binary table
NAXIS1  =                  320 / width of table in bytes
NAXIS2  =                  141 / number of rows in table
PCOUNT  =                    0 / size of special data area
GCOUNT  =                    1 / one data group (required keyword)
TFIELDS =                   80 / number of fields in each row

[NB: from here onwards needs altering for the full WFCAM enhanced 80
attribute set]

TTYPE1  = 'No.     '           / label for field   1
TFORM1  = '1E      '           / data format of field: 4-byte REAL
TUNIT1  = '        '           / physical unit of field
TTYPE2  = 'Isophotal_flux'     / label for field   2
TFORM2  = '1E      '           / data format of field: 4-byte REAL
TUNIT2  = 'Counts  '           / physical unit of field
TTYPE3  = 'Total_flux'         / label for field   3
TFORM3  = '1E      '           / data format of field: 4-byte REAL
TUNIT3  = 'Counts  '           / physical unit of field
TTYPE4  = 'Core_flux'          / Fitted flux within 1x core radius
TFORM4  = '1E      '           / data format of field: 4-byte REAL
TUNIT4  = 'Counts  '           / physical unit of field
TTYPE5  = 'X_coordinate'       / label for field   5
TFORM5  = '1E      '           / data format of field: 4-byte REAL
TUNIT5  = 'Pixels  '           / physical unit of field
TTYPE6  = 'Y_coordinate'       / label for field   6
TFORM6  = '1E      '           / data format of field: 4-byte REAL
TUNIT6  = 'Pixels  '           / physical unit of field
TTYPE7  = 'Gaussian_sigma'     / label for field   7
TFORM7  = '1E      '           / data format of field: 4-byte REAL
TUNIT7  = 'Pixels  '           / physical unit of field
TTYPE8  = 'Ellipticity'        / label for field   8
TFORM8  = '1E      '           / data format of field: 4-byte REAL
TUNIT8  = '        '           / physical unit of field
TTYPE9  = 'Position_angle'     / label for field   9
TFORM9  = '1E      '           / data format of field: 4-byte REAL
TUNIT9  = 'Degrees '           / physical unit of field
TTYPE10 = 'Peak_height'        / label for field  10
TFORM10 = '1E      '           / data format of field: 4-byte REAL
TUNIT10 = 'Counts  '           / physical unit of field
TTYPE11 = 'Areal_1_profile'    / label for field  11
TFORM11 = '1E      '           / data format of field: 4-byte REAL
TUNIT11 = 'Pixels  '           / physical unit of field
TTYPE12 = 'Areal_2_profile'    / label for field  12
TFORM12 = '1E      '           / data format of field: 4-byte REAL
TUNIT12 = 'Pixels  '           / physical unit of field
TTYPE13 = 'Areal_3_profile'    / label for field  13
TFORM13 = '1E      '           / data format of field: 4-byte REAL
TUNIT13 = 'Pixels  '           / physical unit of field
TTYPE14 = 'Areal_4_profile'    / label for field  14
TFORM14 = '1E      '           / data format of field: 4-byte REAL
TUNIT14 = 'Pixels  '           / physical unit of field
TTYPE15 = 'Areal_5_profile'    / label for field  15
TFORM15 = '1E      '           / data format of field: 4-byte REAL
TUNIT15 = 'Pixels  '           / physical unit of field
TTYPE16 = 'Areal_6_profile'    / label for field  16
TFORM16 = '1E      '           / data format of field: 4-byte REAL
TUNIT16 = 'Pixels  '           / physical unit of field
TTYPE17 = 'Areal_7_profile'    / label for field  17
TFORM17 = '1E      '           / data format of field: 4-byte REAL
TUNIT17 = 'Pixels  '           / physical unit of field
TTYPE18 = 'Areal_8_profile'    / label for field  18
TFORM18 = '1E      '           / data format of field: 4-byte REAL
TUNIT18 = 'Pixels  '           / physical unit of field
TTYPE19 = 'Core1_flux'         / Fitted flux within 1/2x core radius
TFORM19 = '1E      '           / data format of field: 4-byte REAL
TUNIT19 = 'Counts  '           / physical unit of field
TTYPE20 = 'Core2_flux'         / Fitted flux within sqrt(2)x core radius
TFORM20 = '1E      '           / data format of field: 4-byte REAL
TUNIT20 = 'Counts  '           / physical unit of field
TTYPE21 = 'Core3_flux'         / Fitted flux within 2x core radius
TFORM21 = '1E      '           / data format of field: 4-byte REAL
TUNIT21 = 'Counts  '           / physical unit of field
TTYPE22 = 'Core4_flux'         / Fitted flux within 2sqrt(2)x core radius
TFORM22 = '1E      '           / data format of field: 4-byte REAL
TUNIT22 = 'Counts  '           / physical unit of field
TTYPE23 = 'RA      '           / label for field  23
TFORM23 = '1E      '           / data format of field: 4-byte REAL
TUNIT23 = 'RADIANS '           / physical unit of field
TTYPE24 = 'DEC     '           / label for field  24
TFORM24 = '1E      '           / data format of field: 4-byte REAL
TUNIT24 = 'RADIANS '           / physical unit of field
TTYPE25 = 'Classification'     / label for field  25
TFORM25 = '1E      '           / data format of field: 4-byte REAL
TUNIT25 = 'Flag    '           / physical unit of field
TTYPE26 = 'Statistic'          / label for field  26
TFORM26 = '1E      '           / data format of field: 4-byte REAL
TUNIT26 = 'N-sigma '           / physical unit of field
TTYPE27 = 'Blank   '           / label for field  27
TFORM27 = '1E      '           / data format of field: 4-byte REAL
TUNIT27 = 'Blank   '           / physical unit of field
TTYPE28 = 'Blank   '           / label for field  28
TFORM28 = '1E      '           / data format of field: 4-byte REAL
TUNIT28 = 'Blank   '           / physical unit of field
TTYPE29 = 'Blank   '           / label for field  29
TFORM29 = '1E      '           / data format of field: 4-byte REAL
TUNIT29 = 'Blank   '           / physical unit of field
TTYPE30 = 'Blank   '           / label for field  30
TFORM30 = '1E      '           / data format of field: 4-byte REAL
TUNIT30 = 'Blank   '           / physical unit of field
TTYPE31 = 'Blank   '           / label for field  31
TFORM31 = '1E      '           / data format of field: 4-byte REAL
TUNIT31 = 'Blank   '           / physical unit of field
TTYPE32 = 'Blank   '           / label for field  32
TFORM32 = '1E      '           / data format of field: 4-byte REAL
TUNIT32 = 'Blank   '           / physical unit of field
EXTNAME = 'APM-BINARYTABLE'    / name of this binary table extension
DATE    = '2003-03-05T10:28:46' / file creation date (YYYY-MM-DDThh:mm:ss UT)
SKYLEVEL=              1880.99 / Median sky brightness (counts/pixel)
SKYNOISE=                 4.54 / Pixel noise at sky level (counts)
THRESHOL=                 9.07 / Isophotal analysis threshold (counts)
MINPIX  =                    5 / Minimum size for images (pixels)
CROWDED =                    1 / Crowded field analysis flag (0 none, 1 active)
RCORE   =                  3.5 / Core radius for default profile fit (pixels)
SEEING  =             2.061935 / Average FWHM (pixels)
COMMENT   FITS (Flexible Image Transport System) format is defined in Astronomy
COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H
INHERIT =                    T
.
.
.
ELLIPTIC=           0.06848837 / Average stellar ellipticity (1-b/a)
CLASSIFD=                    T / Class flag: -1 stellar, 1 non-stellar, 0 noise
SATURATE=               30000. / Average saturation level in frame
APCORPK =             1.852217 / Stellar aperture correction - peak height
APCOR1  =            0.3795509 / Stellar aperture correction - core1 flux
APCOR   =           0.08592701 / Stellar aperture correction - core flux
APCOR2  =           0.03884411 / Stellar aperture correction - core2 flux
APCOR3  =           0.01241398 / Stellar aperture correction - core3 flux
APCOR4  =                   0. / Stellar aperture correction - core4 flux
COMMENT Symbolic translation for GAIA ellipse plotting........
SYMBOL1 = '{Ellipticity Position_angle Areal_1_profile Classification} {el'
SYMBOL2 = 'lipse blue (1.0-$Ellipticity) $Position_angle+90 {} $Classific'
SYMBOL3 = 'ation==1} {sqrt($Areal_1_profile*(1.0-$Ellipticity)/3.142)} : {'
SYMBOL4 = 'Ellipticity Position_angle Areal_1_profile Classification} {el'
SYMBOL5 = 'lipse red (1.0-$Ellipticity) $Position_angle+90 {} $Classific'
SYMBOL6 = 'ation==-1} {sqrt($Areal_1_profile*(1.0-$Ellipticity)/3.142)} :'
SYMBOL7 = '{Ellipticity Position_angle Areal_1_profile Classification} {el'
SYMBOL8 = 'lipse green (1.0-$Ellipticity) $Position_angle+90 {} $Classifi'
SYMBOL9 = 'cation==0} {sqrt($Areal_1_profile*(1.0-$Ellipticity)/3.142)}'
HISTORY 20030305 10:28:46
HISTORY    $Id: cir_classify.c,v 1.3 2003/02/03 09:32:36 jim Exp $
END


ACRONYMS & ABBREVIATIONS

ADnn : Applicable Document No. nn
CASU : Cambridge Astronomical Survey Unit
DAS : Data Acquisition System
ESO : European Southern Observatory
FITS : Flexible Image Transport System
GCS : Galactic Clusters Survey (UKIDSS)
GPS : Galactic Plane Survey (UKIDSS)
GridFTP : Grid File Transfer Protocol
HDS : Hierarchical Data System
HDU : Header Data Unit (FITS nomenclature)
JAC : Joint Astronomy Centre
LAS : Large Area Survey (UKIDSS)
MEF : Multi-Extension FITS
NDF : N-dimensional Data Format
RAID : Redundant Array of Inexpensive Disks
SDSS : Sloan Digitial Sky Survey
VDFS : VISTA Data Flow System
UKIDSS : UKIRT Deep Infrared Sky Survey
UKIRT : United Kingdom Infrared Telescope
VISTA: Visible and Infrared Survey Telescope for Astronomy
WCS : World Co-ordinate System
WFAU : Wide Field Astronomy Unit (Edinburgh)
WSA : WFCAM Science Archive
2MASS : 2 Micron All-Sky Survey


APPLICABLE DOCUMENTS


AD01 WSA Hardware Design Document [9] VDF-WFA-WSA-006

Issue: 1.0, 2/04/03

AD02 WSA Database Design Document [10] VDF-WFA-WSA-007

Issue: 1.0, 2/04/03

AD03 UKIDSS Proposal [13]  
AD04 ESO Data Interface Control Document [1] GEN-SPE-ESO-19940-794

Issue: 2.0

AD05 WSA Overview Document [2] VDF-WFA-WSA-001

Issue: 1.0, 1/04/03



CHANGE RECORD


Issue Date Section(s) Affected Description of Change/Change Request Reference/Remarks
Draft 1 17/03/03 All New document
Draft 2 26/03/03 All First iteration
1.0 2/04/03   First issue (for CDR)
2.0 10/07/03 3.3; 5.2; Refs & Apps New FITS file info


NOTIFICATION LIST

The following people should be notified by email whenever a new version of this document has been issued:


WFAU: P Williams, N Hambly
CASU: M Irwin, J Lewis
QMUL: J Emerson
ATC: M. Stewart
JAC: A. Adamson
UKIDSS: S Warren, A Lawrence

About this document ...

This document was generated using the LaTeX2HTML translator Version 2K.1beta (1.47)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -notransparent -white -split 0 VDF-WFA-WSA-004-I2

The translation was initiated by Nigel Hambly on 2003-07-10


next_inactive up previous
Nigel Hambly 2003-07-10