HDX Data Freshness Database Clean
Project description
Utility to clean Freshness Database
This script cleans the freshness database.
Usage
python -m hdx.freshness.dbactions [-db/--db_uri=] [-dp/--db_params=] [action]
Either db_uri or db_params must be provided or the environment variable DB_URI
must be set. db_uri or DB_URI are of form:
postgresql+psycopg://user:password@host:port/database
db_params is of form:
database=XXX,host=X.X.X.X,username=XXX,password=XXX,port=1234, ssh_host=X.X.X.X,ssh_port=1234,ssh_username=XXX, ssh_private_key=/home/XXX/.ssh/keyfile
action:
-
"clone" which creates a shallow clone of the database which only has all the runs and one dataset and its resources per run for testing purposes.
-
"clean" (the default) cleans the database by removing runs according to these rules:
- Keep a handful of runs around the end of each quarter all the way back to the first run in 2017
- Keep daily runs going back 2 years
- Keep weekly runs from 2 to 4 years back
- Keep monthly runs for 4 years back and earlier
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for hdx-data-freshness-dbclean-1.0.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4fee60397c49b11487f084200805a0102ab85db738507f5baabc76bf884378e3 |
|
MD5 | 9ee752689c0758d165afecdb92e5590a |
|
BLAKE2b-256 | 12cb9053a2a84627f20e9023d9db5eb4b98dfe8ffe3bb53bccf1600f4f4b73bc |
Hashes for hdx_data_freshness_dbclean-1.0.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | df65361b9ba25a31cffcb3529cd8b17e375b77bb6432439eb51aa47aed0cee65 |
|
MD5 | 7ee0f0f9b1e41a6dee8af8808545fc76 |
|
BLAKE2b-256 | 0f98e15633fd667b866d68758780156249950cda42277b5cfcf6f8d27d631ae2 |