Package invocations :: Package cu4 :: Module cu4 :: Class Cu4
[hide private]

Class Cu4

source code


Invoke single frame source detections ingestion given a log of files transferred from the data processing centre.

Nested Classes [hide private]
  NoFilesError
Exception in case the given programme has no catalogues to process.

Inherited from wsatools.DbConnect.IngCuSession.IngCuSession: IngCuError

Instance Methods [hide private]
 
__init__(self, curator='rsc', database='WSA', csvSharePath=False, programmeList=False, progOrder=False, detectionSubset=False, excludeFiles=False, isTrialRun=False, ReDo=False, keepWorkDir=False, xferlog=None, comment='A useful comment')
Initialises data members for all database state flags and details for this curation session.
source code
 
_initFromDB(self)
Initialize constants from the DB
source code
 
_onRun(self)
Do CU4.
source code
 
checkCreated(self, progID)
Check that all files are created.
source code
 
cleanUpFileShare(self)
Make sure there are no old .dat files in the dbSharePath.
source code
 
createIlluminationTableList(self)
Create the list of used illumination tables and write it for use with exnumeric to the working directory.
source code
 
createSchemaDict(self)
Create schema dictionary for every table.
source code
 
splitList(self, p, n)
Split list p into sublists of length n.
source code
 
createSplitListFiles(self, catalogueDict, splitNum=2)
Split the catalogues of each programme into splitNum sublists, where larger catalogues are paired with smaller catalogues.
source code
defaultdict(int, list(str))
getProgIDs(self)
Reads FITS files to determine the set of programme IDs for the catalogues that will be ingested in this curation event.
source code
 
parseIllumTable(self, illumTableName)
Parse the CASU illumination table file into a csv file.
source code
 
runExtractions(self, progID, subFileList)
Run the C++ code to extract the detections from the catalogues.
source code
 
setTableVariables(self)
Set the variables dependend of the tables used in this cu.
source code

Inherited from wsatools.DbConnect.IngCuSession.IngCuSession: __del__, attributesFromArguments, createFileList, createLogFileName, createMonthlyDetSchema, createWorkingDir, getDetTable, getErrorFileName, getMultiframeIDs, getProcessOrder, getProgramIDs, makeSysCmd, removeWorkingDir, run, runSysCmd, setupDetectionTable, testForProgs, translateProgID, writeErrorFiles, writeIngestList

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Static Methods [hide private]

Inherited from wsatools.DbConnect.IngCuSession.IngCuSession: createFilePrefix

Class Variables [hide private]
  maxSplit = 2
  cuedFileExists = False
hash(x)

Inherited from wsatools.DbConnect.IngCuSession.IngCuSession: beginDateDef, endDateDef, keepWorkDir, reqWorkDir, sysc

Instance Variables [hide private]
Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__init__(self, curator='rsc', database='WSA', csvSharePath=False, programmeList=False, progOrder=False, detectionSubset=False, excludeFiles=False, isTrialRun=False, ReDo=False, keepWorkDir=False, xferlog=None, comment='A useful comment')
(Constructor)

source code 

Initialises data members for all database state flags and details for this curation session. Writes a database lock and opens a connection. Also initiates a Programme table object.

Parameters:
  • curator (str) - Name of curator.
  • comment (str) - Descriptive comment as to why curation task is being performed.
  • database (str) - Name of the database to connect to.
  • detectionSubset (list(str)) - Process subset of [Astrometry table, Photometry table, Raw table].
  • excludeFiles (list(str)) - Excluded files from processing.
  • keepWorkDir (bool) - Don't remove working directory.
  • programmeList (list(str)) - Only process data for given programmes (accepts keywords 'all', 'ns' (non-survey), 'ukidss' (all 5 main surveys)).
  • progOrder (list(str)) - The order of processing of the programmes, all programmes not explicitely named can be put anywhere in the list as 'others'.
  • ReDo (bool) - If True, overwrite existing monthly detection schema.
  • xferlog (str) - Logfile containing files to be ingested.
  • isTrialRun (bool) - If True, do not perform database modifications.
Overrides: object.__init__

_onRun(self)

source code 

Do CU4.

Overrides: wsatools.DbConnect.IngCuSession.IngCuSession._onRun

getProgIDs(self)

source code 

Reads FITS files to determine the set of programme IDs for the catalogues that will be ingested in this curation event.

Returns: defaultdict(int, list(str))
A programme ID -> catalogues dictionary.