Background

The Visible and Infrared Survey Telescope for Astronomy (VISTA) is a UK Universities initiative to produce the World's leading survey facility for near-IR (and eventually optical) astronomy. VISTA, scheduled to be operational on Paranal by the end of 2006, will consist initially of a 4-m diameter telescope with a wide field NIR (1.65 degree diameter field of view) imager containing 16 detectors each with 2k$\times$2k pixels.

The UKIRT Wide Field Camera (WFCAM) scheduled to be operational three years before VISTA, by the end of 2003, shares many of the science aims of VISTA, and equally, many of the data processing and archiving problems. With its array of 4 NIR detectors of 2k$\times$2k pixels, WFCAM, notwithstanding its own highly significant science programme and data handling issues, can be thought of as a trialling system for an integrated VISTA data flow system (VDFS).

The Joint Astronomy Centre (JAC) in Hawaii have formally asked the UK wide field astronomy units to devise a suitable scheme for coping with the data stream from WFCAM. Given the similarity in the data processing and archiving requirements of VISTA and WFCAM and the timescales, it is clear that WFCAM can be considered as the prototype for VISTA thereby allowing characterisation of the real behaviour of the system two to three years before the VISTA DFS has to be complete.

CASU and WFAU members have already held a series of meetings with the VISTA and WFCAM project teams to facilitate cooperative development of a DFS capable of dealing with the output and the science requirements of these facilities. The commonality between the projects has lead to the identification of several workpackages that share almost identical aims. In particular the need for: summit pipelines that assess data quality on-the-fly and provide feedback to survey planning and progress tools; basic calibration and science product pipelines to be run in the UK and a subset of these, in the case of VISTA, to be run at ESO HQ; advanced survey product pipeline development to deal with the complex issues of asynchronous image processing including deep stacking; and an active GRID-enabled science archive capable of dealing with and enabling sophisticated datamining queries and of being a cornerstone of the UK Virtual Observatory.

Specifically the agreement with ESO over VISTA depends on the following deliverables to ESO:

• providing modules to ESO for the ESO-run Quality Control pipeline at the telescope;

• providing modules to ESO for the ESO-run data calibration pipeline in Garching;

• delivering agreed WFCAM survey products to the general ESO community as part of the UK contributions in kind;

and also requires the collaborative development of optimised survey planning tools for operation by ESO using innovative feedback of derived quality control information.

These aims are almost identical to WFCAM requirements for: survey planning and progress; a summit pipeline; and a basic data calibration pipeline. Delivering WFCAM science data products in an agreed form has considerable leverage but involves negligible extra development costs.

Furthermore both WFCAM and VISTA have additional overall science-driven requirements for the common development of a system capable of:

• producing generic agreed science survey products from the calibrated frames;

• ingesting the calibrated frames and science survey products into a UK database archive;

• development of a dynamic advanced processing system for combinatorial operations on basic survey products and calibrated frames;

• ensuring the tools to mine the archive of survey products are available through the AstroGrid project.

The end-to-end WFCAM project is overseen by JAC at Hawaii, and in broad outline consists of:

• building and installing the camera and designing instrumentation software (including mountain-top quick-look pipeline) - responsibility ATC/JAC/CASU;
• producing the UK-side data processing pipeline - responsibility CASU;
• providing the end-user science archive - responsibility WFAU.

For the equivalent tasks for VISTA, the Vista Consortium Director has asked CASU and WFAU to be be responsible for at least the Phase A deliverables outlined in the VISTA GSC application. The anticipation is that the considerable overlap in specification and interests between the projects will correspondingly lead to common use of developed software and hardware solutions.

truecm

The remainder of this section of the document is concerned with an overview of the specification requirements for pipeline processing and science data products (science archiving for WFCAM is the responsibility of WFAU and will be dealt with in their submission) both for WFCAM and insofar as we know them for VISTA. For the sake of brevity many of the technical details have been ommitted from this RG2 but are available as part of the WFCAM preliminary design review documentation submitted for the December 2001 UKIRT board meeting, UKIRTB(01)31, and as part of the VISTA DFS documentation, GSC(02)03, submitted to the Grid Steering Committee in March 2002.