Package helpers

source code

Irregular curation task scripts.

Requires: wsatools

Submodules

[hide private]

helpers.CheckCatalogues: Checks for uniqueness of the primary key values in the catalogue FITS files.
helpers.CheckFitsHeaders: Check the fits headers against CASU files and add those missing with data from the database (use with pyfits >3.1.0).
helpers.ConvertEsoAdpFiles: Creates the FITS files necessary for the current ESO-SAF data release requirements.
helpers.CopyTable: Copy table from one DB to another one (with different name)
helpers.CreateReleaseFileList: Calculates constants for the Filter table.
helpers.FitsDiff: Checks differences in metadata of two FITS files.
helpers.FixIngestLogs: Script to fix ingest logs.
helpers.OverlappingFileFinder: Fills the .
helpers.ProgrammeCoverageChecker: Utility class to check that all multiframes in a survey are within a product group and all used to make stacks (unless there is a limit on a stack or different microstepping is used)
helpers.TestFitsVerify
helpers.UpdateConfIDs: Update confID entries for data with different versions.
helpers.UpdateOSAFixedData: Script to fix illum-fixed OSA FITS files that have been released by CASU after catalogue ingests have been completed:
helpers.UpdateTilePawPrintTables: Fills the ppErrBits attribute of the detection tables.

Helper scripts

helpers.AddColumns: Add new attributes to a database table and update their values, or existing attributes, from the original FITS headers values using the prescription defined in the schema file.
helpers.AddIndices: Create table indexes.
helpers.BinaryToCsv: Converts a binary ingest file to human readable csv format.
helpers.CalcMagLmt: Tests new stacks and catalogues of given programme and database release.
helpers.CheckFitsFiles: Reads MultiframeIDs and Timestamps from all FITS files for a given date and report missing values.
helpers.CheckPhotometryHeaders: Checks the photometry data to make sure everything has been marked up correctly.
helpers.CleanUpPixelServer: Creates a shell script to clear file space and delete interleave component files.
helpers.CreateFitsAttributeList: Creates a basic Multiframe SQL Schema file from all fits files in a given directory.
helpers.DeprecateJpegs: Move jpegs of FITS files in deprecated FITS dirs to the deprecated_jpegs dir and update the DB accordingly.
helpers.DeprecateReprocessedStacks: Deprecates stacks in given directory that CASU have reprocessed for Nottingham, but have not yet been ingested into the WSA.
helpers.DiskSetup: Sets up a new disk, creating subdirs ingest, products etc.
helpers.EsoRelease: Creates the FITS files necessary for the current ESO-SAF data release requirements.
helpers.FilterTableCalculator: Calculates constants for the Filter table.
helpers.FitsSchemaCheck: Compares the header contents of the given FITS catalogue file (and the corresponding image file) to the archive schema to spot new keywords.
helpers.FixFitsFiles: Script to fix problematic FITS files:
helpers.GetFileSizes: Determines the used disk space per fits file type.
helpers.ProductAssigner: Utility class for assigning a single multiframe to a product to override deep stacking.
helpers.ProvenanceFiller: Updates the Provenance table.
helpers.ReadFitsKeys: Reads given Fits key(s) from all FITS files for a given date and reports value counts.
helpers.RedoFiles: Upload files from CASU, deprecate existing ones, update the databases and create a list for CU2 to 4.
helpers.RemoveFitsKeys: Removes FITS keywords from FITS headers of files in a given file list.
helpers.ResetProvenanceKeys: Replaces wrong Provenance keywords in a list of FITS files.
helpers.ResetStacks: Remove arithmetic changes to stacks used to create CASU catalogues.
helpers.RollbackIngest: Rolls-back a CU3 (only for now) ingest quickly by ignoring cascading deletes to detections.
helpers.SetupProgramme: Takes Multiframe info to calculate RequiredStacks for new survey.
helpers.SpotDuplicateMFs: Identifies properties of duplicate multiframe entries in the database.
helpers.SyncDb: Fills an empty database with the latest data in the WSA/VSA.
helpers.SyncFlatFileLookUp: Fills or updates the FlatFileLookUp table from Multiframe.
helpers.TestProductionFrames: Tests production frame IDs of all FITS files against entries in the database.
helpers.TransposeCsv: Transposes given CSV file.
helpers.UpdateCatData: Updates database table values from catalogue file headers for new attributes, given attributes or where there is missing/dodgy for a given date.
helpers.UpdateDetectionTables: A general script for updating all detection tables.
helpers.UpdateFitsFileNames: Updates fileName and catName in the given databases from the DB 'disk??' value to the given disk.
helpers.UpdateJpegs: Update JPEG names in the database given a file list.
helpers.UpdateQC: Performs quality control on surveys not covered by NonSurveyRelease nor own QC scripts.
helpers.UpdateQualityFlags: Fills the ppErrBits attribute of the detection tables.
helpers.UpdateStacks: Updates stacks from catalogues with CASUs perl script.
helpers.UpdateTableData: Update database tables with new data from a file for given date range.
helpers.UpdateWSATimestamp: Update/synchronise the filetimestamps (WSATimestamp) with CASU's FITS files (via monitoring/lsCasuDirs.py).
helpers.makelist: Make file lists of one or all fits directories.

Variables

[hide private]

__package__ = None
hash(x)