orac020-schr Version: 01 Original: 1 February 1999 Modified: 22 February 1999
This document sets out a specification and list of requirements for the Observation Scheduler, part of the Observatory Control System, part of the ORAC (Observatory Reduction and Acquisition Control) project.
This document is intended to outline the specification of the Scheduler task in the UKIRT Observation Control System and provide a list of requirements. The requirements list will consist of both user requirements and software requirements. It is intended to be read by anyone with an interest in the UKIRT Observatory Control system. It is assumed that the reader is familiar with the ORAC project, and in particular the OCS component. An overview can be found in [O-12].
This document only describes typical scenarios and lists requirements for scheduling software at UKIRT. The case for flexible scheduling of UKIRT and associated issues are not made here, but are being studied by Andy Adamson, John Davies and UKIRT operations.
This document will briefly describe the relationship of the Scheduler to the rest of the UKIRT system and the system context will be summarised. The major functions of the system are outlined, and the use of the Scheduler software in various scenarios is discussed. Based on these a list of specific user and software requirements will be given.
The Observation Scheduler is part of the UKIRT Observatory Control System, which in turn is one part of the UKIRT Observatory Reduction and Acquisition Control (ORAC) project. References [O-1] and [O-12] give an overview of this project and summarise the relationships.
There is really no predecessor to the Scheduler in the existing UKIRT system. Similar systems are being designed contemporaneously by the Gemini Project [G-18], the ESO VLT [X-17], and the JCMT ([X-15],[X-16]). Nightly scheduling algorithms developed at NASA-Ames [X-18] have been used successfully on some UKIRT service observing nights by John Davies, but they were not linked to the acquisition and control, or to a Phase-2 observing tool output.
The functions required may be thought of as those required for scheduling a night or more of observing (Observation Scheduler) and those required for longer term scheduling and planning on timescales of weeks, months, semesters (Strategic Scheduler). Together these functions are to be provided by the Scheduler.
In summary the main functions of the Observation Scheduler are:
The main functions of the Strategic Scheduler are:
The Scheduler must run at the UKIRT telescope for interactive observing (in service modes, queue-scheduled modes and classical observing modes) but part of it must also be able to run anywhere as an aid to planning a nights observing. Portability is desired as is remote access to the Observation Database. The strategic scheduling functions need to be available at the UKIRT TAG meetings.
The Scheduler will interact with the Observation Database and with the Chooser. When in use to drive observing the Scheduler will supplant the Science Program/Observation Selection part of the Chooser in determining an observation to be executed. The translation, sequencer control and user interface functions of the Chooser will remain. The strategic scheduling portion may also interact with Phase-1 level documents.
Set out in this section are a few scenarios which describe how the most common uses of the Scheduler are envisioned to happen. Details of the process of acquiring data (taking the observation) and reducing it are omitted - they may be found in documents describing the relevant systems.
It is assumed in all cases that there exists at the telescope an Observation Database consisting of Science Programs submitted by Observers (Principal Investigators or their collaborators). The Observations for these Science Programs contain all the necessary details - instrument configurations, observing sequences, co-ordinates, required conditions, etc. The details of how these Science Programs arrive in the database are dealt with elsewhere [O-19].
This scenario covers the case where an Observer wishes to plan the night's observing during the day before. This could be a staff astronomer about to carry out queue-scheduled observing or a visiting observer who wishes to plan their own observing to achieve the best science and efficiency...
This describes how the system would be used at the telescope by a staff astronomer carrying out queue-scheduled observing.
This describes how a visiting observer would use the system to carry out their own observing. In fact this scenario is essentially that presented in Section 3.2, so we will not detail it here. The main difference is that the visiting observer only has access to Science Programs and Schedule Plans that they are authorised to use. Other than that it is intended that the tools are the same. Thus the "classical" observer may completely ignore the Scheduler and carry out observations one at a time via the Chooser, or they may make use of the full functionality of the Scheduler to optimise their observing.
The classical observer may wish to use the Scheduler to draw up nightly schedules at their home institute, before travelling to Hawai'i.
The following scenarios illustrate how UKIRT service and re-active scheduling can be handled by the Scheduler without requiring any changes to current UKIRT practice. UKIRT may or may not continue to use these modes, but the Scheduler software and ORAC system should not force changes onto operations.
Although allocation of Service time is a separate process, Service Observing will fit into the general scheme in one of two general ways. If, as at present, individual Service nights are inserted into the semester at regular intervals successful Service programmes may be prepared using the OT and submitted to the observing database flagged as being of type "service". The Service observer can then use the Scheduler as described in 3.1 to set up a queue of observations for execution on the service night. If Service programmes are treated as short queue-scheduled programmes then they are simply added to the database and carried out with the other programmes as described in 3.2.
At present there is a method for re-actively rescheduling high priority "classical" programmes which are affected by poor conditions. If the process of allocating classical nights with reactive status were to continue but UKIRT were to be operating mixed queue/classical semesters, then it can be accommodated by allocating a block of nights to the recovery time, and converting them into queue nights if they are not required. Alternatively changing the classification of the programme from classical to queue, with an appropriate priority and completion information could be used if only a small amount of additional time were needed.
It is assumed in what follows that in a given semester there will be a mix of programmes allocated classical blocks of time and programmes that are to be flexibly scheduled as queues. For example, engineering work will still need allocated nights and the wide field surveys will need large blocks of time, even if time with the other instruments is fully flexibly scheduled. Exactly what this mix is, or the reasons for the mix, should not matter to the design.
A long term scheduling tool will be needed during the time allocation process to keep track of the "loading" of various parts of the semester with both classical and queue scheduled programmes. This implies that the Strategic Scheduler be able to work from "phase1" level input for co-ordinates, estimated observing times, and possibly environmental conditions for each target. Awarded time information will be entered by the UKIRT schedule manager as the TAG meeting progresses.
Once the allocation process is over the tool may be used to draw up an initial Semester Plan with blocks of queue-scheduled nights and classical blocks, based on a set of "rules" relating to items such as mean Ra for classical runs, priority, instrument availability, weather statistics.
A key difference between the Observation Scheduler and the Strategic Scheduler when dealing with flexibly scheduled programmes is in the level of detail. Whereas the Observation Scheduler is concerned with detailed scheduling of a nights observing, the Strategic Scheduler has to ensure that the distribution of target positions, required time, required conditions and priority is consistent with the time available and expected conditions, given the constraints of the "classical" blocks. If the Observation Scheduler "rules" are appropriate, then once the general pattern has been set for the semester by the Strategic Scheduler nightly use of the Observation Scheduler should ensure that the flexibly scheduled programmes are carried out as the TAG intended. Any "adjusting" that is necessary (e.g. to raise the chances of a low priority almost completed programme being done) should be accomplished by changing the parameters/weights for the Observation Scheduler.
The scenarios above describe the use of the Scheduler to plan a night of queue-scheduled observing, or plan out a semester of allocations. For intermediate timescales the Observation Scheduler may be used to create night by night detailed plans. i.e. it will have a mode whereby having scheduled a given Observation on a given night, it will then disregard it (as if it had been completed) when calculating the Observing Plans for subsequent nights. The Observation scheduler will allow Observing Plans for a few (~3) consecutive nights to be calculated using the same environmental conditions and parameter weights for all of them. The user may input an appropriate mix of likely weather conditions for the next several nights, to get a view of how a queue-scheduled observing period is likely to progress. This can be done for any look ahead period, with best guesses at good/bad conditions. The Strategic Scheduler will be able to use saved Observing Plans as nights to schedule, as well as Science Programmes as defined by the OT for classical programmes, and engineering nights. Thus if a number of Observing Plans have been generated for a number of nights/weeks the Strategic Scheduler may be used to provide an over-view of the period. Graphical tools to aid information digest will be needed.
This is not strictly a function of the Scheduler, rather of the OCS as a whole, although programme/observation completion information needs to be read from the database by the scheduler. The short description below is included for completeness.
The system will "keep track" of observations completed on queue scheduled nights, and enter information about completed observations into the Observing Database. Queue scheduled observers and staff will be able to query the database to obtain information about the status of programmes. UKIRT staff may need to manually change the completion status of observations within a programme, or of the whole programme, to allow for problems with the data discovered after the observing run is over. For many simple programmes it may be possible to add data quality assessment to the ORAC-DR recipes to enable automation of basic checking or highlight questionable data for UKIRT staff to examine. The issues relating to the database management are discussed in [O-19].
These requirements are drawn from the document written by AJA and JKD describing general requirements for flexible scheduling on UKIRT, to which specific software requirements have been added based on the Scenarios above.
FM1. The Scheduler shall read relevant information from the Science Programs and Observation Definitions in the Observation Database and use a series of hierarchical "rules" to draw up an "Observing Plan" for a night of observing.
FM2. The hierarchy of criteria to be applied in drawing up a plan for a given night will include, but not be limited to:
1. Requested instrument and instrument configuration is available
2. Programme priority
3. Source availability, or source availability function = availability (of 1 source or rms hour angle distribution, or...) x priority
4. Expected Conditions and real conditions
5. Programme Completion level
6. Instrument schedulability - all else being equal, rare instruments first
7. Required instrument setup (e.g. don't keep swapping spectroscopy wavelengths)
8. Instrument selection consistency
9. Multiple telescopes /simultaneity required
10. Programme urgency
The details of how such rules will be applied is tbd during detailed design in consultation with UKIRT staff, and may depend on the choice of scheduling engine.
FM3. The weighting applied to criteria must be changeable
FM4. The user may pre-select programmes/observations for inclusion in an Observing Plan and/or exclude specific programmes/observations.
FM5. The Observing Plan and parameters used to create it will be saved before execution.
FM6. In creating an Observing Plan the Scheduler will calculate the duration of observations based on the information in the Observation Definition, and a simple assumed overhead.
FM7. The Scheduler must identify the need for and check adequacy of calibration observations. Exactly how this is to be done is tbd.
FM8. In creating an Observing Plan the Scheduler will schedule grouped and chained observations appropriately.
FM9. User may modify the plan by changing criteria, or excluded programmes/observations.
FM10. If user requests, scheduler will update plan to account for changed real-time condition of weather or instruments.
FM11. If more than one instrument is used for a given nights plan the Scheduler must be able to handle the scheduling of calibrations that do not require the telescope at an appropriate time.
FM12. There must be some way to ensure that there are no "gaps" in the Schedule due to inability of the Scheduler to schedule Observations that match the criteria it was given because no such observations exist.
FM13. When in use during observing the user may recalculate the nights Observing Plan at any time.
FM14. Status of progress through the nights plan to be available to observer/TO
FM15. Scheduler will pass to Chooser the name of the next observation definition to be carried out.
FM16. Unless the Plan is re-calculated by the observer the Scheduler will assume that the current one is to be used.
FM17. When in use during observing the Scheduler will monitor conditions (weather, time) and warn user of significant changes - i.e. those that mean the current conditions no longer match those required for the currently executing observation.
FM18. When the next observation is requested by the Chooser, the Scheduler will check the time, and if necessary skip to the most appropriate observation in the currently executing queue.
FM19. At any time during observing the Chooser may be set back to manual (and hence the queue over-ridden).
FM20. Strategic scheduler should keep track of relative "loading" of different parts of the semester with classical and flexible allocations, based on Phase 1 level inputs and awards of time.
FM21. Strategic scheduler to use rules similar to those in FM2 to draw up long term schedule with blocks of flexible, classical and engineering nights.
FM22. Strategic Scheduler will allow some events to be on "fixed" nights.
FM23. Strategic scheduler may be used to update long term schedule if changes are required, e.g. cancelled engineering nights.
FM24. Scheduler will ignore completed programmes and observations when drawing up a plan for a night or a semester.
SM1. The User must only have access to Observation Definitions he or she is authorised to view and use. Exactly what this means needs to be defined.
SM2. The TAG chairman will need access to the status information in the database too.
This is a list of the documentation requirements for the system. Similar requirements are placed on other parts of the ORAC project, and the numbering is the same where they are duplicated.
DO1. A programmers guide is required both in html and on paper.
DO2. A user guide is required in html and a paper copy.
DO3. User guide to be accessible from "help" in GUI.