Home       Trees       Indices       Help   
WSA
Package wsatools :: Module SystemConstants :: Class SystemConstants
[hide private]

Class SystemConstants

source code

Constants common to the entire VDFS system.

Instance Methods [hide private]

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __init__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __subclasshook__

    Pixel server common directories
list(str)
downloadPathList(self)
Returns: Complete list of paths on all RAID discs where the data will be downloaded to.		 source code
    FITS file naming conventions
list(int)
extListOfType(self, productType='stack', frameType=None)
Returns: List of expected extensions in given product or database frame type.		 source code
str
productTypeOfFrame(self, frameType)
Returns: Product type of given database frame type.		 source code
list(str)
wfauFitsKeys(self)
Returns: List of FITS header keys inserted into files by WFAU.		 source code
    CASU data transfer constants
str
scpMethod(self, light=True)
Returns: Command to run scp.		 source code
str
sshMethod(self, light=True)
Returns: Command to run ssh.		 source code
str
scpMainDir(self)
Returns: Top directory of the data tree on the scp server.		 source code
    Convenience methods
 
__str__(self)
Converting a object of this class type to a string returns the classes name.		 source code
bool
isGES(self)
Returns: True, if these are GES constants.		 source code
bool
isSSA(self)
Returns: True, if these are SSA constants.		 source code
bool
isWSA(self)
Returns: True, if these are WSA constants.		 source code
bool
isVSA(self)
Returns: True, if these are VSA constants.		 source code
bool
isOSA(self)
Returns: True, if these are VSA constants.		 source code
    Getter methods
str
dbMntPath(self, loadServer=None)
Returns: Path to the NFS share mounted on the catalogue load server.		 source code
str
calibrationSchema(self)
Returns: Name of the calibration tables schema SQL script file.		 source code
str
curationSchema(self)
Returns: Name of the curation tables schema SQL script file.		 source code
str
ingestedLogsPath(self, sharePath=None)
Returns: Path to the directory of ingest log files on the NFS share of the catalogue load server (Windows).		 source code
str
ingestedFilesPath(self, sharePath=None)
Returns: Path to the directory of files to be ingested on the NFS share of the catalogue load server (Windows).		 source code
str
ingestListsPath(self, sharePath=None)
Returns: Path to the directory of ingest file lists on the NFS share of the catalogue load server (Windows).		 source code
str
metadataSchema(self)
Returns: Name of the metadata tables schema SQL script file.		 source code
str
wcsErrorFilesPath(self)
Returns: Path to the directory of files with WCS errors on the NFS share of the catalogue load server (Windows).		 source code
    Software directory structure
str
calibrationDir(self)
Returns: Directory for calibration log files.		 source code
str
cambInfoPath(self)
Returns: Path to camb info??		 source code
str
corruptFilesPath(self)
Returns: Path for corrupted files.		 source code
str
filterDataDirPath(self)
Returns: Full path to the directory containing filter data		 source code
str
filterRefDataDirPath(self)
Returns: Full path to the directory containing filter reference data		 source code
str
illuminationDir(self)
Returns: Directory for illumination table files.		 source code
str
logFilePath(self, fileName='')
Returns: Path to curation logs.		 source code
str
missingFilesFilePath(self)
Returns: Path to log of missing or broken files.		 source code
str
monitorPath(self)
Returns: Path to the monitor code directory.		 source code
str
nonSurveyRegPath(self, fileName='')
Returns: Path of the non-survey resgistration files		 source code
str
tempWorkPath(self)
Returns: Path to temporary working directory for curation scripts.		 source code
str
xferLogPath(self)
Returns: Path of the transfer log files.		 source code
    Pixel server directory structure
list(str)
availableRaidFileSystem(self)
Returns: Disks which are available for storage.		 source code
list(str)
excludedRaidFileSystem(self)
Returns: Disks which are excluded from the mass storage raid file system.		 source code
int
leaveSpaceOnMSRFS(self, byPercent=False)
Space to be left free on the disks in the mass storage raid file system.		 source code
list(str)
massStorageRaidFileSystem(self)
Returns: A list of paths to every mass storage raid disk.		 source code
    Permanent curation log file paths
str
filesBrokenPath(self)
Returns: Path to the log listing files that are broken in some way, e.g.		 source code
str
filesDuplicatedPath(self)
Returns: Path to the log listing files that have duplicate non-deprecated entries in the database.		 source code
str
filesMissingPath(self)
Returns: Path to the log listing files with database entries, but not existing at the location provided by the database entry.		 source code
str
filesNotIngestedPath(self)
Returns: Path to the log listing files not ingested (i.e.		 source code
    Web site constants
str
surveyBaseUrl(self)
Returns: URL to the science archive web interface.		 source code
str
surveyJspBaseUrl(self)
Returns: URL to the science archive web interface including port.		 source code
Static Methods [hide private]
    FITS file naming conventions
list(str)
fileNameSuffices()
Returns: A list of file name suffices ordered by precedence.		 source code
dict(str:str)
frameTypeToSuffix()
Returns: A dictionary of file name suffices referenced by frame type.		 source code
    Initialiser
SystemConstants
__new__(cls, switchName=None)
Creates a new object of type SystemConstants or its subclass if the supplied argument suggest a specific subclass is more appropriate.		 source code
    Getter methods
 
getAll(constantName)
Returns the values of the given constant in all subclasses.		 source code
str
getServerName(fsPath)
The name of the server that hosts the supplied disk path.		 source code
list(str)
catServerVolumes(server='')
List of logical volumes on the given catalogue server.		 source code
str
dbSharePath(fileName='', serverName='neferefre')
Returns: Full path to the given file located on the samba share.		 source code
    Software directory structure
str
casuCodePerlPath()
Returns: Path to CASU Perl scripts directory.		 source code
str
curationCodePath()
Returns: Path to curation source code directory.		 source code
str
emptyFitsCataloguePathName()
Returns: Full path to the default empty catalogue file.		 source code
str
helpersPath()
Returns: Path to the helpers code directory.		 source code
str
sExtractorInputsPath()
Returns: Path to SExtractor inputs directory.		 source code
str
sourcePosTablesPath()
Returns: Path where Cu16 works & stores source position tables.		 source code
str
sqlFragmentPath()
Returns: Path to sql script fragments used by SqlWrappers.py.		 source code
str
sqlNonSurveyPath(fileName='')
Returns: Path to non-survey sql scripts.		 source code
str
sqlQualityControlPath(fileName='')
Returns: Path to quality control sql scripts.		 source code
str
sqlTemplatePath(fileName='')
Returns: Path to sql script templates.		 source code
str
sqlMonthlyDetPath(fileName='', full=True)
Returns: Path to monthly detection schemas.		 source code
str
testOutputPath(fileName='')
Returns: Path to permanent test directory used in examing test results.		 source code
str
privNetCmd(server, cmd)
Returns: SSH command to be run on remote WFAU curation server.		 source code
Class Variables [hide private]
  obsCal = ObsCal()
Observation dates for the survey.
    Numerical constants
  minNumCPUs = 6
Minimum number of CPUs for tiling software.
  one_kilobyte = 1024
A kilobyte in bytes.
  one_megabyte = 1048576
A megabyte in bytes.
  one_gigabyte = 1073741824
A gigabyte in bytes.
  sqlDataTypeSize = {'bigint': 8, 'datetime': 8, 'float': 8, 'in...
Dictionary of SQL data type sizes in bytes referenced by data type name.
    Catalogue server constants
  adminDatabase = 'master'
The name of the database context within which to perform miscellaneous administrative commands (e.g.
  catServerFileSep = '\\'
Path separator character for the catalogue server's O/S.
  catServerSharePath = '\\\\NEFEREFRE\\share\\'
Share path to curation server as seen by the catalogue server.
  clusterName = 'ramses'
Root name of the cluster servers.
  loadServerVolumes = ('G', 'H', 'I')
Default load server volume configuration (for cluster servers).
    Pixel server file system
  curationServers = ['thoth', 'menkaure', 'shepseskaf', 'userkaf...
Available curation servers
  criticalDisks = ['/disk01/wsa', '/disk02/wsa', '/disk05/wsa', ...
Disks running a dump system backup or moved to other disks.
  deprecatedDataStorage = ['/disk06/wsa', '/disk08/wsa', '/disk0...
Disks on which development takes place, so no other data will be stored there.
  developDisks = ['/disk47/sys', '/disk34/vsa', '/disk34/wsa', '...
Disks on which development takes place, so no other data will be stored there.
  infinibandServers = ['ini', 'neferefre', 'kakai', 'sahure', 'u...
Curation servers with infiniband connections.
  sysPath = '/disk01/wsa/sys'
System path for lock and shell scripts.
  userInterfaceTempStorage = ['/disk07/wsa']
Disks on which development takes place, so no other data will be stored there.
    Pixel server common directories
  casu2massDir = '/disk47/sys/Viz2Mass/fits'
Directory containing CASU 2mass catalogues for wcsfit
  compressImDir = 'products/jpgs'
Directory for compressed images (jpgs).
  deprecatedComprImDir = 'products/deprecated_jpgs'
Path to directory for deprecated compressed images (jpgs).
  deprecatedDir = 'ingest/deprecated'
Directory for deprecated files.
  diffDir = 'products/diffs'
Directory for difference image products.
  downloadDir = 'ingest/dload'
Final top directory where the data will be stored.
  extProdDir = '/disk47/sys/ExternalProducts'
Directory for external products.
  esoExtDir = '/disk47/sys/ExternalProducts/teamCats'
Directory for external team catalogues
  fitsDir = 'ingest/fits'
Final top directory where the data will be stored.
  headerDir = 'ingest/headers'
Final top directory where the header metadata will be stored.
  listDrivDir = 'products/list'
Directory for list driven catalogue products.
  mosaicDir = 'products/mosaics'
Directory for mosaic image products.
  productLogDir = 'products/logs'
Directory for production log files.
  productsDir = 'products'
Directory for WFAU produced files, ie.
  stackDir = 'products/stacks'
Directory for stacked image products.
  swarpDir = '/disk47/sys/ExternalProducts/swarpMos'
Directory for external SWARP mosaics
  tempProductsDir = 'products/temp'
Directory for temporary products.
  tileDir = 'products/tiles'
Directory for tiled image products.
    FITS file naming conventions
  wfauPrefix = 'e'
Prefix for WFAU produced FITS files.
  darkPrefix = 'dark_'
Prefix for dark frame FITS files.
  skyPrefix = 'sky_'
Prefix for sky frame FITS files.
  biasPrefix = 'bias_'
Prefix for dark frame FITS files.
  catSuffix = '_cat'
Suffix for catalogue FITS files.
  fringeSuffix = '_fringe'
Suffix for fringe FITS files.
  confSuffix = '_conf'
Suffix for confidence image map FITS files.
  cpmSuffix = '_cpm'
Suffix for CPM(?) FITS files.
  deepSuffix = '_dp'
Suffix for deep stacked FITS files.
  diffSuffix = '_diff'
Suffix for science frame FITS files.
  filtSuffix = '_two'
Suffix for filtered FITS files.
  filtPrefix = 'tmp_'
Prefix for filtered FITS files.
  flatSuffix = '_flat'
Suffix for flat frame FITS files.
  leavSuffix = '_sf'
Suffix for science frame FITS files.
  listSuffix = '_list'
Suffix for list-driven catalogue FITS files.
  mjdSuffix = '_mjd'
Suffix for temporary MJD map FITS file.
  mosaicSuffix = '_mos'
Suffix for mosaiced FITS files.
  pregrtSuffix = '_pgrt'
Suffix for pre-grouted tile catalogue FITS files.
  simpleSuffix = '_simple'
Suffix for simple? FITS files.
  stackSuffix = '_st'
Suffix for stacked frame FITS files.
  fixcatSuffix = '_fix_cat'
Suffix for OSA fixed catalogue FITS files.
  tileSuffix = '_tl'
Suffix for tiled FITS files.
  catType = '.fits'
Extension type for catalogue FITS files.
  jpgType = '.jpg'
Extension type for compressed JPEG thumbnail image files.
  mefType = '.fit'
Extension type for multi-extension FITS files.
  uncType = '.unc'
Extension type for uncompressed types.
  frameTypes = ['mosaic', 'tile', 'deep', 'filt', 'leav', 'diff'...
A list of frame types ordered by precedence in a combined frame type.
    Software file structure
  cirdrShellScript = '/neferefre/scos/usr/local/wsa/cirdr/cir_st...
Full path to the shell script to set up the correct environment for CASU's cirdr package of source extractor related software.
  emptyFitsCatalogueFileName = 'empty_catalogue.fits'
Name of the default empty catalogue file.
  procScript = 'COMMON_StoredProcsAndFuncs.sql'
Script for generic procedures and functions for all databases.
  sqlScriptPath = '/neferefre/rsc/VDFS_branches/trunk/sql'
Path to sql scripts.
  srcPath = '/neferefre/rsc/VDFS_branches/trunk/src'
Path to present sandbox's source code.
  webDocsPath = '/neferefre/rsc/WFAUweb/webpages'
Path to web pages.
    SExtractor
  defaultSexInparFile = 'defaultDetCat.inpar'
Default sex inpar file.
    CASU data transfer constants
  ftp_method = 'ftp'
Command to run ftp
  rem_camlist_dir = '/home/wfcam'
Directory at CASU for list files.
  rem_lock_name = 'readlock'
Remote lock file name.
  rem_ready2copy_name = 'OK_TO_COPY'
Remote ready to copy file name.
  rem_success_name = 'SUCCESSFULLY_READ'
Remote(=local) success file name.
  scpThreads = 1
Number of parallel threads to use for scp.
  scpUser = 'wfcam'
The user login name for the CASU scp server.
    Synoptic syntax constants
  uncSubStr = 'UNC'
Substring representing uncorrelated synoptic table.
  corSubStr = 'COR'
Substring representing correlated synoptic table.
  bthSubStr = 'BOTH'
Substring representing both synoptic tables.
  varSubStr = 'VAR'
Substring representing variability table.
    Common instrument properties
  channelWidth = 128
Channel width.
  instrumentXnPix = 2048
Number of pixels in detector (X).
  instrumentYnPix = 2048
Number of pixels in detector (Y).
  hasMicroStep = False
has microstepping?
  hasVarOrient = False
has variable orientation?
  hasOffsetPos = False
has offset positions?
  hasEsoRelease = False
Has an ESO release?
Properties [hide private]

Inherited from object: __class__

Method Details [hide private]

__new__(cls, switchName=None)
Static Method

source code 

Creates a new object of type SystemConstants or its subclass if the supplied argument suggest a specific subclass is more appropriate.

Parameters:
  • switchName (SystemConstants or str) - Initialised SystemConstants object or else a string that contains necessary data to switch on, e.g. database name, the INSTRUME FITS header keyword, full path to a FITS file etc.
Returns: SystemConstants
Overrides: object.__new__

__str__(self)
(Informal representation operator)

source code 

Converting a object of this class type to a string returns the classes name.

Overrides: object.__str__

isGES(self)

source code 
Returns: bool
True, if these are GES constants.

isSSA(self)

source code 
Returns: bool
True, if these are SSA constants.

isWSA(self)

source code 
Returns: bool
True, if these are WSA constants.

isVSA(self)

source code 
Returns: bool
True, if these are VSA constants.

isOSA(self)

source code 
Returns: bool
True, if these are VSA constants.

getServerName(fsPath)
Static Method

source code 

The name of the server that hosts the supplied disk path.

Parameters:
  • fsPath (str) - Path to anywhere within a disk file system.
Returns: str

catServerVolumes(server='')
Static Method

source code 

List of logical volumes on the given catalogue server.

Parameters:
  • server (str) - Optionally supply name of the server, otherwise assumes it's a standard ahmose-like configuration.
Returns: list(str)

dbSharePath(fileName='', serverName='neferefre')
Static Method

source code 
Returns: str
Full path to the given file located on the samba share.

dbMntPath(self, loadServer=None)

source code 
Returns: str
Path to the NFS share mounted on the catalogue load server.

calibrationSchema(self)

source code 
Returns: str
Name of the calibration tables schema SQL script file.

curationSchema(self)

source code 
Returns: str
Name of the curation tables schema SQL script file.

ingestedLogsPath(self, sharePath=None)

source code 
Returns: str
Path to the directory of ingest log files on the NFS share of the catalogue load server (Windows).

ingestedFilesPath(self, sharePath=None)

source code 
Returns: str
Path to the directory of files to be ingested on the NFS share of the catalogue load server (Windows).

ingestListsPath(self, sharePath=None)

source code 
Returns: str
Path to the directory of ingest file lists on the NFS share of the catalogue load server (Windows).

metadataSchema(self)

source code 
Returns: str
Name of the metadata tables schema SQL script file.

wcsErrorFilesPath(self)

source code 
Returns: str
Path to the directory of files with WCS errors on the NFS share of the catalogue load server (Windows).

calibrationDir(self)

source code 
Returns: str
Directory for calibration log files.

cambInfoPath(self)

source code 
Returns: str
Path to camb info??

casuCodePerlPath()
Static Method

source code 
Returns: str
Path to CASU Perl scripts directory.

corruptFilesPath(self)

source code 
Returns: str
Path for corrupted files.

curationCodePath()
Static Method

source code 
Returns: str
Path to curation source code directory.

emptyFitsCataloguePathName()
Static Method

source code 
Returns: str
Full path to the default empty catalogue file.

filterDataDirPath(self)

source code 
Returns: str
Full path to the directory containing filter data

filterRefDataDirPath(self)

source code 
Returns: str
Full path to the directory containing filter reference data

helpersPath()
Static Method

source code 
Returns: str
Path to the helpers code directory.

illuminationDir(self)

source code 
Returns: str
Directory for illumination table files.

logFilePath(self, fileName='')

source code 
Returns: str
Path to curation logs.

missingFilesFilePath(self)

source code 
Returns: str
Path to log of missing or broken files.

monitorPath(self)

source code 
Returns: str
Path to the monitor code directory.

nonSurveyRegPath(self, fileName='')

source code 
Returns: str
Path of the non-survey resgistration files

sExtractorInputsPath()
Static Method

source code 
Returns: str
Path to SExtractor inputs directory.

sourcePosTablesPath()
Static Method

source code 
Returns: str
Path where Cu16 works & stores source position tables.

sqlFragmentPath()
Static Method

source code 
Returns: str
Path to sql script fragments used by SqlWrappers.py.

sqlNonSurveyPath(fileName='')
Static Method

source code 
Returns: str
Path to non-survey sql scripts.

sqlQualityControlPath(fileName='')
Static Method

source code 
Returns: str
Path to quality control sql scripts.

sqlTemplatePath(fileName='')
Static Method

source code 
Returns: str
Path to sql script templates.

sqlMonthlyDetPath(fileName='', full=True)
Static Method

source code 
Returns: str
Path to monthly detection schemas.

tempWorkPath(self)

source code 
Returns: str
Path to temporary working directory for curation scripts.

testOutputPath(fileName='')
Static Method

source code 
Returns: str
Path to permanent test directory used in examing test results.

xferLogPath(self)

source code 
Returns: str
Path of the transfer log files.

privNetCmd(server, cmd)
Static Method

source code 
Returns: str
SSH command to be run on remote WFAU curation server.

availableRaidFileSystem(self)

source code 
Returns: list(str)
Disks which are available for storage.

excludedRaidFileSystem(self)

source code 
Returns: list(str)
Disks which are excluded from the mass storage raid file system.

leaveSpaceOnMSRFS(self, byPercent=False)

source code 

Space to be left free on the disks in the mass storage raid file system.

Parameters:
  • byPercent (bool) - If True, free disc space is defined by percentage rather than absolute values.
Returns: int
Free space available.

massStorageRaidFileSystem(self)

source code 
Returns: list(str)
A list of paths to every mass storage raid disk.

downloadPathList(self)

source code 
Returns: list(str)
Complete list of paths on all RAID discs where the data will be downloaded to.

filesBrokenPath(self)

source code 
Returns: str
Path to the log listing files that are broken in some way, e.g. fail FITS verification checks or have missing header keywords such as the PROV keyword.

filesDuplicatedPath(self)

source code 
Returns: str
Path to the log listing files that have duplicate non-deprecated entries in the database.

filesMissingPath(self)

source code 
Returns: str
Path to the log listing files with database entries, but not existing at the location provided by the database entry.

filesNotIngestedPath(self)

source code 
Returns: str
Path to the log listing files not ingested (i.e. not in database, and possibly not on the pixel servers' raid array).

fileNameSuffices()
Static Method

source code 
Returns: list(str)
A list of file name suffices ordered by precedence.

frameTypeToSuffix()
Static Method

source code 
Returns: dict(str:str)
A dictionary of file name suffices referenced by frame type.

extListOfType(self, productType='stack', frameType=None)

source code 
Returns: list(int)
List of expected extensions in given product or database frame type.

productTypeOfFrame(self, frameType)

source code 
Returns: str
Product type of given database frame type.

wfauFitsKeys(self)

source code 
Returns: list(str)
List of FITS header keys inserted into files by WFAU.

scpMethod(self, light=True)

source code 
Returns: str
Command to run scp.

sshMethod(self, light=True)

source code 
Returns: str
Command to run ssh.

scpMainDir(self)

source code 
Returns: str
Top directory of the data tree on the scp server.

surveyBaseUrl(self)

source code 
Returns: str
URL to the science archive web interface.

surveyJspBaseUrl(self)

source code 
Returns: str
URL to the science archive web interface including port.

Class Variable Details [hide private]

sqlDataTypeSize

Dictionary of SQL data type sizes in bytes referenced by data type name.

Value:
{'bigint': 8,
 'datetime': 8,
 'float': 8,
 'int': 4,
 'real': 4,
 'smallint': 2,
 'tinyint': 1}

adminDatabase

The name of the database context within which to perform miscellaneous administrative commands (e.g. attach/detach etc). For Microsoft SQL Server, this database context is "master".

Value:
'master'

curationServers

Available curation servers

Value:
['thoth',
 'menkaure',
 'shepseskaf',
 'userkaf',
 'sahure',
 'neferefre',
 'ini',
 'unas']

criticalDisks

Disks running a dump system backup or moved to other disks.

Value:
['/disk01/wsa',
 '/disk02/wsa',
 '/disk05/wsa',
 '/disk06/wsa',
 '/disk10/wsa',
 '/disk33/vsa']

deprecatedDataStorage

Disks on which development takes place, so no other data will be stored there.

Value:
['/disk06/wsa', '/disk08/wsa', '/disk09/wsa']

developDisks

Disks on which development takes place, so no other data will be stored there.

Value:
['/disk47/sys', '/disk34/vsa', '/disk34/wsa', '/disk34/osa']

infinibandServers

Curation servers with infiniband connections.

Value:
['ini',
 'neferefre',
 'kakai',
 'sahure',
 'userkaf',
 'shepseskaf',
 'menkaure']

productsDir

Directory for WFAU produced files, ie. stacked, mosaiced, and difference images.

Value:
'products'

frameTypes

A list of frame types ordered by precedence in a combined frame type.

Value:
['mosaic',
 'tile',
 'deep',
 'filt',
 'leav',
 'diff',
 'stack',
 'conf',
...

cirdrShellScript

Full path to the shell script to set up the correct environment for CASU's cirdr package of source extractor related software.

Value:
'/neferefre/scos/usr/local/wsa/cirdr/cir_startup.bash'

srcPath

Path to present sandbox's source code. Useful if working on a different branch of the source code.

Value:
'/neferefre/rsc/VDFS_branches/trunk/src'

defaultSexInparFile

Default sex inpar file. Modelled on UDS-DR8.

Value:
'defaultDetCat.inpar'

   Home       Trees       Indices       Help   
WSA
Generated by Epydoc 3.0.1 on Mon Sep 8 15:46:56 2014 http://epydoc.sourceforge.net