Package wsatools :: Package DbConnect :: Module CuSession :: Class CuSession

Class CuSession

A curation session. Any data curation activity for the archive where a log is required to be recorded, should either be designed as a class that inherit from this class or else an instance of this class should be invoked by that curation script. This class design ensures a log is correctly written, errors are correctly handled, temporary files are correctly managed and that the database ArchiveCurationHistory and ProgrammeCurationHistory tables are updated if required. Also a database connection, temporary working directory, and programme table objects are provided if required.

Warning: The destructor won't be called if the instance of a CuSession object is copied to a member variable that stores it (if it used only temporarily then that's OK - either the member variable uses the reference to the CuSession temporarily or the reference to the CuSession is passed to a temporary variable). This is because a circular reference occurs on destruction.

Note: In the destructor you cannot order the destruction of the archive and log objects by just calling del. Therefore, I've left their order of destruction undefined - the order is not important.

Nested Classes

[hide private]

Nested Errors and Exceptions

CuError
A curation error instead of a programming error.

Instance Methods

[hide private]

__init__(self, programmeID='', curator='rsc', comment='A descriptive comment', database='WSA', isTrialRun=False, userName='ldservro', cli=None)
Writes a database lock and opens a connection. source code

__del__(self)
Finalises curation session - updates the ArchiveCurationHistory and ProgrammeCurationHistory tables, if required.

source code

run(self)
Executes the _onRun() method of the curation session within a try block, so that all the standard exceptions may be dealt with correctly.

source code

logBrokenFile(self, fileDetails, logFilePath=None)
Appends the details of the broken file to end of the provided log file.

source code

_onRun(self)
Curation sessions that inherit from this class should override this method, executing within all of the tasks that required of this curation session.

source code

_onException(self)
Attempts to rollback transaction and sets the database state flags, so that the destructor realises the curation session failed.

source code

_getNextCuEventID(self)
Obtain the next available cuEventID.

source code

int

_getNextID(self, attrName, tableName, where='')
Obtain next available ID number in the db for a given ID attribute. source code

_scrubACHPlaceHolder(self)
Scrubs the placeholder default row in the ArchiveCurationHistory table, generated by _getNextCuEventID().

source code

_updateArchiveCurationHistory(self)
Adds an entry for this curation session to the ArchiveCurationHistory database table.

source code

_updateProgrammeCurationHistory(self)
Adds an entry for this curation session to the ProgrammeCurationHistory database table.

source code

_prepareDbLock(self)
Writes database lock.

source code

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Static Methods

[hide private]

str

logException(error)
Logs the given trapped exception.

source code

Class Variables

[hide private]

_numSessions = 0

eTypes = set([])
Exception types trapped by any CuSession in this process.

sysc = SystemConstants()
System constants for this CuSession.

cuNum = -9999
Curation task number.

isDayStampedLog = False
Include day in log file name?

_autoCommit = False
Should this curation task auto-commit database transactions?

_cleanUpDirs = True
Always clean-up all temporary files.

_isPersistent = True
Should this curation task try to re-open broken database connections?

_reqDbLock = True
Should only a single instance of this curation task run at a time?

_reqCuEventID = False
In trial run mode is a cuEventID required?

_useWorkDir = False
Create a temporary work directory.

comment = 'A descriptive comment'
Descriptive comment as to why curation task is being performed.

curator = 'rsc'
Name of curator.

archive = None
A connection to the archive being curated (a DbSession object).

cuEventID = None
Unique ID for this curation event.

onlyNonSurveys = False
Only curate non-survey programmes?

onlySurveys = False
Only curate survey programmes?

programme = None
A DataFactory.ProgrammeTable - quick access to Programme table data.

programmeID = None
ID, or set of IDs, of (all) the programme being curated this session.

resultsFilePathName = 'NONE'
Results file for ArchiveCurationHistory.

shareFileID = ''
Unique ID tag for files on the catalogue load server share directory.

_cuTable = None
The DataFactory.CurationTaskTable - quick access to CurationTask.

_lockPathName = None
Location of the database lock file.

_log = None
The curation log - an instance of a Logger class.

_rolledBack = None
Has the CU transaction rolled-back?

_success = None
Has the programme being curated successfully?

Properties

[hide private]

Inherited from object: __class__

Method Details

[hide private]

init(self, programmeID=`''`, curator=`'rsc'`, comment=`'A descriptive comment'`, database=`'WSA'`, isTrialRun=False, userName=`'ldservro'`, cli=None)
(Constructor)

source code

Writes a database lock and opens a connection. Obtains a unique curation event ID and prepares a Logger object to hold the verbose record of this curation event, whilst reserving a placeholder in the database ArchiveCurationHistory to log the record of this curation event. A temporary working directory is created, if required, and all member variables are initialised.

Parameters:

programmeID (str) - Optionally supply the unique identifier of a programme or survey name initials to curate selected programme. Set to 'NONSURVEYS' if just curating non-surveys or 'SURVEYS' for surveys only.
curator (str) - Name of curator.
comment (str) - Descriptive comment as to why curation task is being performed.
database (str) - Name of the database to connect to.
isTrialRun (bool) - If True, do not perform database modifications, just print the SQL statement to the terminal.
userName (str) - Optionally override default database username.
cli (CLI.CLI) - Optionally initialise using the settings supplied by the command-line options.

Overrides: object.__init__

del(self)
(Destructor)

source code

Finalises curation session - updates the ArchiveCurationHistory and ProgrammeCurationHistory tables, if required. Cleans-up any temporary files and files in the catalogue server share directory. The database lock is removed, whilst the connection is automatically dropped, as well as the log being automatically written a file.

Note: Order of deletion of self.archive and self._log is random, so the disconnect from database message isn't also saved to the log file, but it will always be displayed.

run(self)

source code

Executes the _onRun() method of the curation session within a try block, so that all the standard exceptions may be dealt with correctly.

To Do: Replace init and del with a context manager for database connections and locks, implemented as a with-block in this function.

logException(error)
Static Method

source code

Logs the given trapped exception.

Parameters:

error (Exception) - Exception caught.

Returns: str

Derived error type (for full list see CuSession.eTypes).

logBrokenFile(self, fileDetails, logFilePath=None)

source code

Appends the details of the broken file to end of the provided log file. Ensures that file has correct permisions if it does not already exist. If in test mode the broken file is not logged.

Parameters:

fileDetails (str) - Details of broken file, at least the file name.
logFilePath (str) - Full-path to the log file.

_onException(self)

source code

Attempts to rollback transaction and sets the database state flags, so that the destructor realises the curation session failed. Can override in subclasses to tidy up run tasks specific to that CU following an exception, but always call this method.

_getNextCuEventID(self)

source code

Obtain the next available cuEventID. Whilst putting a placeholder row into the ArchiveCurationHistory table to reserve this cuEventID, preventing it from being hijacked by a parallel process.

_getNextID(self, attrName, tableName, where=`''`)

source code

Obtain next available ID number in the db for a given ID attribute.

Parameters:

attrName (str) - Name of ID attribute.
tableName (str) - Name of the table where attribute resides.
where (str) - Optional SQL WHERE clause.

Returns: int

Next available ID number.

_prepareDbLock(self)

source code

Writes database lock. If lock already present then pause and wait for database to unlock.

Class CuSession

__init__(self, programmeID='', curator='rsc', comment='A descriptive comment', database='WSA', isTrialRun=False, userName='ldservro', cli=None) (Constructor)

__del__(self) (Destructor)

run(self)

logException(error) Static Method

logBrokenFile(self, fileDetails, logFilePath=None)

_onException(self)

_getNextCuEventID(self)

_getNextID(self, attrName, tableName, where='')

_prepareDbLock(self)

init(self, programmeID=`''`, curator=`'rsc'`, comment=`'A descriptive comment'`, database=`'WSA'`, isTrialRun=False, userName=`'ldservro'`, cli=None)
(Constructor)

del(self)
(Destructor)

logException(error)
Static Method

_getNextID(self, attrName, tableName, where=`''`)