Package wsatools :: Module FitsUtils :: Class FitsList

Class FitsList

Generate lists for FITS directories.

Instance Methods

__init__(self, sysc=SystemConstants("WSA"), prefix='')
x.__init__(...) initializes x; see help(type(x)) for signature source code

createFitsDateDict(self, disklist=None, ingestDirectory='ingest/fits', trafoLog=None, beginDateStr='19990101', endDateStr='99991231', versionStr=None, forceLists=False)
Create a dictionary listing all dates for every fits directory in disklist. source code

str

getVersion(self, datumVersStr)
Get the version number from the date directory name, exclude test directories (ending in *_v.comment).

source code

dict

createTrafoLogDict(self, logFileName)
Read the transfer log and create a date sorted dict.

source code

allFiles(self, pattern, searchPath, pathSep=':')
Get all file names matching the given string pattern. source code

getFileCount(self, pattern, searchPath, pathSep=':')
Get the number of files matching the given string pattern. source code

int, int, int

countFileTypes(self, fileList)
Count the files in the given list grouped by file type.

source code

createDailyFileLists(self, workDir, startDate=10000101, endDate=99991231, versionStr='1', trafoLog=None)
List all files in one date directory into one filelist. source code

writeFileLists(self, outDir, datumStr, fitsList)
Write filelists into a working directory for further processing.

source code

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__

Properties

[hide private]

Inherited from object: __class__

Method Details

[hide private]

init(self, sysc=SystemConstants("WSA"), prefix=`''`)
(Constructor)

source code

x.__init__(...) initializes x; see help(type(x)) for signature

Overrides: object.__init__: (inherited documentation)

createFitsDateDict(self, disklist=None, ingestDirectory=`'ingest/fits'`, trafoLog=None, beginDateStr=`'19990101'`, endDateStr=`'99991231'`, versionStr=None, forceLists=False)

source code

Create a dictionary listing all dates for every fits directory in disklist.

Parameters:

beginDateStr (str) - First date of fits date directories.
disklist (list[str]) - List of all disks having the ingest directory.
endDateStr (str) - Last date of fits date directories.
forceLists (bool) - Return a dictionary of lists.
ingestDirectory (str) - The directory containing the fits date directories.
trafoLog (str) - The transfer logfile name.
versionStr (str) - The version of the date directories.

getVersion(self, datumVersStr)

source code

Get the version number from the date directory name, exclude test directories (ending in *_v.comment).

Parameters:

datumVersStr (str) - The date directory name.

Returns: str

Version number.

createTrafoLogDict(self, logFileName)

source code

Read the transfer log and create a date sorted dict.

Parameters:

logFileName (str) - The transfer log file.

Returns: dict

Dictionary containing all information of the transfered files.

allFiles(self, pattern, searchPath, pathSep=`':'`)

source code

Get all file names matching the given string pattern.

Note: Despite using os.listdir() glob is still as fast as dircache.listdir() having done tests of my own to prove it.

countFileTypes(self, fileList)

source code

Count the files in the given list grouped by file type.

Parameters:

fileList (list(str)) - List of FITS files.

Returns: int, int, int

number of pixel and catalogue files (excluding 'e*'), number of pixel files (excluding 'e*'), number of catalogue files (excluding 'e*')

createDailyFileLists(self, workDir, startDate=10000101, endDate=99991231, versionStr=`'1'`, trafoLog=None)

source code

List all files in one date directory into one filelist.

Parameters:

workDir (str) - Working directory path.
startDate (int) - First date to process.
endDate (int) - Last date to process.
versionStr (str) - Version number of the data.
trafoLog (str) - Logfile containing files to be ingested.

writeFileLists(self, outDir, datumStr, fitsList)

source code

Write filelists into a working directory for further processing.

Parameters:

outDir (str) - Working directory path.
datumStr (str) - The date directory name.
fitsList (list(str)) - List of FITS files.

Class FitsList

__init__(self, sysc=SystemConstants("WSA"), prefix='') (Constructor)

createFitsDateDict(self, disklist=None, ingestDirectory='ingest/fits', trafoLog=None, beginDateStr='19990101', endDateStr='99991231', versionStr=None, forceLists=False)

getVersion(self, datumVersStr)

createTrafoLogDict(self, logFileName)

allFiles(self, pattern, searchPath, pathSep=':')

countFileTypes(self, fileList)

createDailyFileLists(self, workDir, startDate=10000101, endDate=99991231, versionStr='1', trafoLog=None)

writeFileLists(self, outDir, datumStr, fitsList)

init(self, sysc=SystemConstants("WSA"), prefix=`''`)
(Constructor)

createFitsDateDict(self, disklist=None, ingestDirectory=`'ingest/fits'`, trafoLog=None, beginDateStr=`'19990101'`, endDateStr=`'99991231'`, versionStr=None, forceLists=False)

allFiles(self, pattern, searchPath, pathSep=`':'`)

createDailyFileLists(self, workDir, startDate=10000101, endDate=99991231, versionStr=`'1'`, trafoLog=None)