Home | Trees | Indices | Help |
|
---|
|
A special type of database session, where you wish to ingest data into a database table from a native binary or CSV file. Upon initialisation this class verifies that the schemas of the database tables are up-to-date.
To Do: Always parse indices on initialisation of Ingester. Can't drop indices at this stage as it should occur too soon. Then give ingestTable() a default option of dropIndices=True. Ingester can tell from the schema whether it should parse non-survey indices or not. Ingester itself could have a dropIndices option at initialisation with default of False, for IngIngester to overrule.
|
|||
IngestError Exception thrown when the data cannot be ingested. |
|||
Errors & Exceptions | |||
---|---|---|---|
Inherited from |
|
|||
|
|||
int |
|
||
str |
|
||
tuple(str, str) |
|
||
|
|||
Inherited from Inherited from Inherited from |
|
|||
fileOnShare = None Name of file on share for IngIngester type ingests only. |
|||
fileTag =
Tag to append to ingest files so they can be cleaned-up later. |
|||
_schema = None Dictionary of schemas for tables to be ingested into, ref by table name. |
|||
Inherited from Inherited from |
|
|||
Inherited from |
|
|||
Inherited from |
|
Makes connection to the requested database, and checks that the schemas of the tables supplied in the table list are correct.
|
Ingest a binary or CSV flat file table into the database.
Note: If the table doesn't already exist in the database it will be created. However, it won't have foreign key constraints (this is a feature intended for release databases made by CU19). If you need foreign key constraints it is best to call createTable() prior to ingestTable(overWrite=False). To Do: If not isOrdered and overWrite=True then we don't apply primary key until after ingest. OR even better - if not isOrdered try to create table without any constraints, if succeed, i.e. there is no existing table or overWrite=True, then apply primary key after ingest other don't but do check foreign keys. May make parts of NeighbourTableIngester redundant. |
Moves a file to the catalogue load server's share directory as mounted on the curation server. Works around the share violation problem under NFS: a file written to the curation-client / load-server NFS file share cannot be accessed immediately after being written unless it is first renamed!
|
Move ingest file to a place visible to the catalogue server O/S and normalise different filePathName inputs into the format: filePathName = Full path to file from the catalogue server O/S e.g. H:\dir\subdirilename.ext fileName = filename.ext or subdir/filename.ext if file is in a subdirectory of the catalogue server share directory.
|
Remove the ingest file.
|
Home | Trees | Indices | Help |
|
---|
Generated by Epydoc 3.0.1 on Mon Sep 8 15:46:51 2014 | http://epydoc.sourceforge.net |