Basic pipelines

The basic pipeline will operate on single pointings or other ``well-defined'' observing sequences (protocols specified in MSBs/OBs) taken during a single night to produce instrument signature-free images and to derive object catalogues from them. The object catalogues will be used to provide the first pass astrometric and photometric calibration and together with the images will define the basic science survey products.

The two-dimensional image data products at this stage will be instrumental signature-corrected: single frames; lossless interleaved super frames; stacked dither (super)frames; tiled contiguous mosaics, combining four pointings with any of previous options; and confidence maps for all pipeline output products (details TBD).

The catalogue data products will be lists of detected images with an agreed set of parameters (details TBD) summarising useful astronomical information and providing the necessary DQC information.¹

In more somewhat more detail the requirements are:

• instrumental signature removal - continuous updating of flatfielding and sky background tracking and correction, if necessary, during interleaving/dither sequence stacking, possible need to also remove other 2D background variations from imperfect multi-quadrant operation of detectors;
• define and produce a strategy for dealing with image retention from preceding exposures if present;
• combine frames if part of an observed dither sequence or tile pattern and develop a consistent internal calibration scheme to put observations on an approximately uniform system to simplify latter external calibration;
• basic catalogue generation including astrometric and photometric calibration, shape and DQC information. This should make use of an appropriate World Coordinate System (WCS) which will enable the basic photometric calibration from the catalogues using suitable pre-selected standard areas covering the entire field-of-view to monitor and control systematics;
• for VISTA, a UK-supplied calibration pipeline will be also be run at ESO HQ in Garching by ESO's DMD, to provide the essential calibration and QC information for their raw Data Archive. ESO's data processing responsibility for their community stops at this point;
• each frame and catalogue supplied with provisional calibration information and overall morphological classification embedded in FITS files and include propagation of error arrays and use of confidence maps;
• processing simulated data and federating with other catalogues to assess the completeness, systematics and accuracy of the science product catalogues;
• the ability to reprocess the entire dataset after upgrades/bug fixes.

Although many of the basic processing modules are in place, we are still evaluating alternative high level control and user interface strategies for overall pipeline flow control. A generic flowchart outlining the main processing tasks required for the basic pipeline is shown in figure 1.

Figure 1: Overview of processing operations required to deliver basic science survey products from WFCAM/VISTA data.