HDX Data Freshness Database Clean
Project description
Utility to clean Freshness Database
This script cleans the freshness database.
Usage
python -m hdx.freshness.dbactions [-db/--db_uri=] [-dp/--db_params=] [action]
Either db_uri or db_params must be provided or the environment variable DB_URI
must be set. db_uri or DB_URI are of form:
postgresql+psycopg://user:password@host:port/database
db_params is of form:
database=XXX,host=X.X.X.X,username=XXX,password=XXX,port=1234, ssh_host=X.X.X.X,ssh_port=1234,ssh_username=XXX, ssh_private_key=/home/XXX/.ssh/keyfile
action:
-
"clone" which creates a shallow clone of the database which only has all the runs and one dataset and its resources per run for testing purposes.
-
"clean" (the default) cleans the database by removing runs according to these rules:
- Keep a handful of runs around the end of each quarter all the way back to the first run in 2017
- Keep daily runs going back 2 years
- Keep weekly runs from 2 to 4 years back
- Keep monthly runs for 4 years back and earlier
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for hdx-data-freshness-dbclean-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 96ec3b239bff1e09ad069166600522949aea32dc5676a69afa75b187c8d6189f |
|
MD5 | 0707cc2263e05aa5a592570d2da6c7e1 |
|
BLAKE2b-256 | e7f72c6540d44b1e0500d8287a8a4eeb2ace8bfc3a3b1b18acf581fab6438f54 |
Hashes for hdx_data_freshness_dbclean-1.0.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee59c1b521a533da22d9fc4421d1abd28d9ccc50810c5e5df69947e1d835d5a0 |
|
MD5 | fa777909276dac4fc42432a74b8f835d |
|
BLAKE2b-256 | ccc47ac28f53ad7f5c1a2f980939ec6b366b5da52f6934d200c1e7724f2ebf5a |