Scenarios

Gillian Wright, Alan Bridger, Frossie Economou

orac002-scen Version: 01 Original: 23 July 1997 Modified: 2 December 1997

This document describes in detail three basic observing sequences and their corresponding reduction recipes. This information can then be used to evaluate prototypes for UDR, Prep and the OCS.

1.0 Introduction

The new observatory control sequence must be able to ensure that the correct data reduction recipe is used for any given observing sequence and the recipes need to carry out the appropriate reduction steps for that sequence. To test the ideas for both UDR and the OCS this document provides a specification for all stages of the acquisition and reduction of data for 3 standard scenarios. These scenarios have been chosen to reflect 3 of the frequently used observing techniques at UKIRT. The scenarios are (1) photometry of a moderately bright point source in the K-band, (2) CGS4 spectroscopy of a point source in the K-band and (3) spectro-polarimetry of point sources. The reduction recipes for observing sequences (1) and (2) reproduce functionality that is currently provided by the "STRED" script in IRCAMDR and the Standard DRConfig in CGS4DR, with the desired improvements that have been listed on various DR software worklists, e.g. that discussed at the December 1996 UKIRT projects meeting. The polarimetry recipe simply reproduces the steps in Antonio's polarimetry reduction script, for one of the recommended polarimetry sequences. (Separate documents will detail and prioritise the various sequences and reduction recipes for imaging with UFTI, and spectroscopy and imaging with the other instruments).

2.0 How it will look to the user

When the user chooses an appropriate observing sequence for their source the Preparation software should advise them of the corresponding data reduction recipe. Some high level error trapping will be required to ensure that if a user chooses a standard observing sequence and data reduction recipe, but then changes the observing sequence in a fundamental way (e.g. removes essential calibration observations) they will be warned that this may "break" the data reduction recipe and that they should choose another recipe or change the standard one into their own appropriate recipe. There are some parameters in a standard observing sequence that may be changed by the user without affecting the steps that the data reduction recipe has to carry out, for example the size of a nod, and the data reduction recipes should be designed to cope with these.

It is a requirement of the new system that calibration frames may be used for several observations of astronomical sources, and even shared between programmes (e.g. a night of K-band spectroscopy all at the same central wavelength will require only a few arcs and flats). Thus some observing sequences will have associated with them a data reduction recipe that assumes the appropriate calibrations have already been obtained as part of an earlier sequence. In this case the user needs some warning that they need to ensure the observations are obtained first, and also the data reduction scripts need to stop cleanly if the user forgets to do so.

3.0 Point Source K-band Photometry

The details of possible standard observing sequences to be used for photometry of point sources have recently been thoroughly examined in the UKIRTDR discussion emails. It is likely that there will be a few different recommended observing sequences each requiring a slightly different reduction recipe, with the choice of observing sequence depending on the approximate brightness of the source. The methods have in common the median filtering of frames to derive a flat, though the frames to be used differ. The scenario described in this section is that which has been common practice at UKIRT until recently, but will probably be superseded by a slightly different observing sequence soon. None-the-less it requires the same "typical" reduction steps as the newer observing sequences, and is thus a good test of the UDR, Prep and OCS concepts.

3.1 Observing Sequence

The observing sequence phot_scenario consists of the following actions

Set the instrument to appropriate K band configuration
Take a dark using this configuration
Slew the telescope to the requested position. This step should happen concurrently with steps 1 and 2.
When the dark has completed and the slew has completed, pause the observing sequence so that the telescope operator can centre the source in the default position on the array. The method used by the TO to do this may vary according to circumstances (e.g. by turning the guider on, having previously aligned IR image at the default array position with the guide spot, by using a quicklook display in the IR and moving the telescope to centre the IR image, etc.) The default position on the array will be a 5arcsec square box at a clean area near the centre of the array. The centre of this box will be a UKIRT recommended location which is known and is generally what will be used.
When the operator has moved the target into position and defined the base position for offsets, resume the observing sequence.
Take a 5-point jitter map of the target using this configuration. A 5-point jitter map consists of the following steps:

· Take an Object frame at the default position

· Offset to the position 8 0

· Take an Object frame at this position

· Offset to the position 0 8

· Take an Object frame at this position

· Offset to the position -8 0

· Take an Object frame at this position

· Offset to the position 0 -8

· Take an Object frame at this position

· Offset to the position 0 0

Start the next observing sequence....

3.2 Data Reduction Recipe

The data reduction recipe dr_phot_scenario corresponding to the observing sequence phot_scenario described above should reduce the data to the point where a photometric magnitude is calculated (without user interaction) which is reliable enough that an observer can use the magnitude so derived to monitor the photometric performance and the quality of the night. This means that it needs to carry out the following data reduction steps:

Identify the dark and check that it has the same on-chip exposure time and the same exposure type ("stare" or "nd-stare") as the first object frame. Report an error message and stop if this is not the case - since it will indicate that the data is not being obtained using a valid phot_scenario observing sequence.
Reduce the dark (this may be a null operation, of course).
Subtract dark frame from each object frame, as it is obtained.
Apply the relevant bad pixel mask to each dark subtracted object frame, as soon as dark is removed. (so UDR must know which array it is e.g. UFTI or IRCAM, UIST or Michelle, also the data may be a sub-array)
When all 5 object frames have been obtained find median value for each pixel, using an algorithm like the "IRAF sigma clipping" version which identifies and excludes deviant values.
Create new data array containing "median" value for each pixel.
Normalise the median data array - the result is the Flat.
Divide each dark subtracted object frame by the Flat
Read the headers of the frames to determine relative Ra, Dec, offsets (this could be done earlier?), this implies that an appropriate offset is available in the header.
Correct the images for rotation of the x-y axes with respect to Ra and Dec. Rotation of array is "known", but could change e.g. after a warm up and so this information needs to be read from (a setup file?, the header?) somewhere.
Shift all of the frames by the appropriate amount to align with the position of the first frame. Ideally steps 10 and 11 would be done together so that only 1 re-binning of the data is required.
Add the shifted frames together.
Divide by the number of frames in the stack.
Auto-display the resulting image for information.
Find the position of the star on the array - software needs to search a 5x5 arcsec box centered at the default position on the array for a source. This means that UDR must be able to know the default position from somewhere. If no source is found stop and report this since either the phot_scenario sequence was not used, or the default position was not used (and the UDR was not informed of the change), or there really is no source detected, in which case you can't do photometry of it.
Report the x,y position of the source found on the screen.
Write this position to a "photometry file" along with the name of the object, airmass, filter, and UT for later use. (There should be one such photometry file per instrument and night).
Calculate brightness of star, in DN/second, and the signal-to-noise. This should be done by using a profile fitting algorithm to fit to the star and surrounding sky. (e.g. DAOPHOT?)
Convert star counts into an instrumental magnitude (-2.5logDN/sec) and report value to the screen.
Write instrumental magnitude into the photometry file
Calibrate the instrumental magnitude using a system zeropoint and canonical correction for airmass to derive a "preliminary reduced magnitude".
Report magnitude with error to screen
Write this preliminary reduced magnitude to the photometry file.

3.2.1 Manual Data reduction steps

Steps 1-22 above complete the automated reduction of photometry. However while the automated reduction of this or the next source is being carried out, there are some manually invoked steps that the observer may want to use:

Plotting of frames with varying plot limits. This is particularly true of the final coadded image.
Check photometry on individual reduced frames to investigate errors, problems etc.
Print to the screen or to a printer the photometry file with the results summary.
Run the (interactive) data reduction script reduce_phot - this is a script which uses the photometry file for the night to calculate zero points, airmass corrections and re-reduce the photometry. It will normally be run at the end of the night, but the observer may like to invoke it at some point during the night.

3.3 Allowed variations

Although the dr_phot_scenario data reduction recipe should have to reduce only data taken using the standard phot_scenario observing sequence, there are some changes to the parameters in the standard observing sequence which the user could make, and which the data reduction recipe should be expected to cope with. Fundamental changes to the style of the observing sequence (e.g. not taking a dark, nodding to blank sky) would mean that this was a new observing sequence, not phot-scenario and so it should have a corresponding new data reduction recipe. The things that may be changed in the phot-scenario observing sequence which the DR should be able to handle are:

The number of positions in the jittering may be increased - but it should always be an odd number > 3 (Jitter maps of only 3 positions will need a different recipe altogether). If an even number of jitter positions is accidentally used, the data reduction script could cope by ignoring the last file (but it is tbd whether this is desirable behaviour).
The size of the offsets may be changed.
The location of the default position may be changed. (e.g. because for some reason the usual area of the array is not stable on the night).

4.0 Point/Small Source K-band Low-Resolution Spectroscopy with CGS4

CGS4 spectroscopy of a point source could be obtained by using 3 separate observing sequences one of which takes calibration data, one observes the ratio-ing standard and one observes the source (this is generally the current practice at UKIRT). An alternative would be to run a single "long" observing sequence which takes all the data. Flats and arcs are always needed, and this requirement needs to be accounted for somehow even if the exposures do not need to be taken. If an entire night is being spent at one wavelength then flats and arcs will typically be taken a few times per night, but not re-observed for every astronomical target. In either case the data on the ratio-ing star would be a separate observation since it is a separate astronomical target. The data reduction steps for the ratio-ing star are the same as those applied to the source, up to the point of ratio-ing. In describing an observing sequence and the corresponding data reduction recipe for point source spectroscopy it is assumed here that a flat, an arc and a ratio star have already been observed. (how this can be assured is still tbd).

Observers using a standard spectroscopy observing sequence will need to be able to chose between a few data reduction recipes that do or do not carry out certain optional steps such as dividing by the ratio-ing star, or subtracting sky lines. The following recipe includes all the options to ensure that it is a good test.

4.1 The observing sequence

The observing sequence spec_scenario consists of the following actions

Set the instrument to appropriate configuration
Slew the telescope to the requested position. This step should happen concurrently with step 1.
When the slew has completed, pause the observing sequence so that the telescope operator can centre the source in the default row on the array. The method used by the TO to do this may vary according to circumstances (e.g. by turning the guider on, having previously aligned IR spectrum on the default row with the guide spot, by using a quicklook display in the IR and moving the telescope to centre the IR image, by running "peakup" or its equivalent, etc.) The default row on the array will be at a UKIRT recommended location which is known and is generally what will be used.
When the operator has moved the target into position and defined the base position for offsets, resume the observing sequence.
Observe an indeterminate number of "slit nodded pairs". Slit nodded pairs consist of the following steps:

· Define the default position to be "offset 0 0"

· Take an Object frame at the default position

· Offset along the slit by 13.6 arcsec

· Take an Object frame at this position

· Offset back to the default (0 0) position

· Take an Object frame at this position

· Repeat this sequence until the user sends a stop request (which may happen at any point in the sequence).

When stop request received, finish the current observation then stop the sequence
Offset back to the (0, 0) position
Start the next observing sequence

4.2 Data Reduction Recipe

The data reduction recipe dr_spec_scenario, should reduce the data obtained using the observing sequence spec_scenario to the point where the user can assess the true signal-to-noise in the spectrum, and identify any features by wavelength having confidence that they are not atmospheric artefacts. To do this it needs to carry out the following data reduction steps:

Identify an appropriate FLAT frame. (This means the DR must be able to check grating, wavelength, order, whether the grating has since moved, time and/or position from the header). If there is no appropriate FLAT stop and return an error message.
Reduce the FLAT frame if this has not already been done previously. Reduction of the flat means:

· Subtract a BIAS if it is of exposure type "stare"

· Normalise FLAT by fitting a second order polynomial along the rows and dividing by it.

If the object frame is of exposure type "stare" subtract a BIAS frame from each of its constituent integration frames.
Divide each of the integration frames by the normalised FLAT
Interleave the integration frames according to the sampling with which they were observed.
Identify an appropriate ARC frame. (same constraints as for identifying flat).
Reduce the ARC frame if this has not been done previously. Reduction of the arc means:

· Carry out steps 3-5 above, using the previously identified FLAT

· Use the wavelength, grating, order, and arclamp information to compare arc spectrum with one in library of CGS4 arcs and thereby identify at least 3 lines.

· If insufficient lines, calculate approximate linear wavelength axis using central wavelength and dispersion information from the header, and flag it as approximate.

· Use lines identified to calculate wavelength scale for spectrum

Apply wavelength scale from appropriate ARC to the data
Plot the result
Wait for the next frame
Check next frame is at a different offset position from the first - report error and stop if this is not the case, since it will indicate that the observing sequence spec_scenario was not used to obtain the data, or something has gone wrong.
Carry out steps 3,4,5, 8, for this frame.
Use the header information about the offset along the slit, the pixel size, and the position of the default row to calculate where on the array the second spectrum will be. (this step only needs to be done once when the first two observations are available).
Remove residual sky lines from each observation using algorithm like Figaro "polysky" with three sky areas defined to be two rows in from the bottom of the frame to 5 rows short of the lower of the two slit positions, 5 rows between the standard positions, and from 5 rows above the higher of the standard positions to 2 rows down from the top edge.
When two frames at different offsets have been obtained and reduced, subtract them in the sense subtract frame at non-zero offset position from the one at offset position 0 0
Co-add this difference into a "running sum of pairs frame"
plot the "running sum of pairs frame"
Identify the appropriate standard for the data. (This means that after a standard has been observed it must be labelled somehow as being a standard, by the user).
Divide the running sum of pairs by the standard and output to a temporary file
Plot the image that results from the division
Optimally extract the two source spectra assuming the source is approximately on the default row and at the row determined from the offset distance along the slit i.e. exact position should be determined before extraction. Number of rows to optimally extract over will be a changeable parameter with a sensible default.
Combine the two optimally extracted spectra and plot the result.
When the next frame arrives start a new pair and wait for the next one, checking the two are indeed at different positions before creating pair.
When this pair is co-added to the "running sum of pairs frame" repeat steps 18-22
When the stop request is noted, determine if there is another pair to process and if so follow steps 3-22 as appropriate.
When the last pair is added keep running sum frame, ratioed frame and extracted spectrum as "permanent" files and tidy up the temporary ones.
Start the reduction of the next set of data.

4.2.1 Manual Data reduction steps

Steps 1-26 above complete the automated reduction of spectroscopy. However while the automated reduction of this or the next source is being carried out, there are some manually invoked steps that the observer may want to use:

Plotting of frames with varying plot limits. This is particularly true of the running sum image and the extracted spectrum.
Flux calibration of image or extracted spectrum
Change the ratioing star in use (!) or decide to skip the ratioing step. Ideally this needs to be achieved without having to stop and start adding all the pairs up again for example. (sometimes you cannot be sure until you look at the data whether on-line ratioing is helpful to assessing data quality or not).
Extraction and plotting of other rows e.g. if the source turns out to have a companion or some extended emission.

4.3 Allowed variations

Although the dr_spec_scenario data reduction recipe should have to reduce only data taken using the standard spec_scenario observing sequence, there are some changes to the parameters in the standard observing sequence which the user could make, and which the data reduction recipe should be expected to cope with. Fundamental changes to the style of the observing sequence (e.g. nodding to blank sky instead of along the slit) would mean that this was a new observing sequence, not spec-scenario and so it should have a corresponding new data reduction recipe. The things that may be changed in the spec-scenario observing sequence which the DR should be able to handle are:

The number of slit nodded pairs may be pre-determined.
The size of the offset along the slit may be changed.
The location of the default row may be changed. (e.g. because for some reason the usual area of the array is not stable on the night).

5.0 Spectro-Polarimetry

This will be specified soon. It will be based on one Antonio Chrysostomou's scripts for reducing data from a standard spectro-polarimetry Execs.