Import tool from GeoNature to a PostgreSQL database through Export module API (client side)
Project description
This project provides an import data from GeoNature instances to a PostgreSQL database (client side). Widely inspired from ClientApiVN
Project Setup
GN2PG Client can be installed by running pip. It requires Python 3.7.4 or above to run.
pip install gn2pg-client
Issues
Please report any bugs or requests that you have using the GitHub issue tracker!
HowTo
Help
gn2pg_cli --help
Init config file
This command init a TOML config file within ~/.gn2pg hidden directory (in user HOME directory), named as you want. PLEASE DO NOT SPECIFY PATH!
gn2pg_cli --init <myconfigfile>
Config file is structured as this. [[source]] block can be duplicate as many as needed (one block for each source).
# GN2PG configuration file
# Local db configuration
[db]
db_host = "localhost"
db_port = 5432
db_user = "<dbUser>"
db_password = "<dbPassword>"
db_name = "<dbName>"
db_schema_import = "schema"
# Additional connection options (optional)
[db.db_querystring]
sslmode = "prefer"
# Source configuration,
# Ducplicate this block for each source (1 source = 1 export)
[[source]]
# Source name, will be use to tag stored data in import table
name = "Source1"
# GeoNature source login
user_name = "<monuser>"
# GeoNature source password
user_password = "<monPwd>"
# GeoNature source URL
url = "<http://geonature1/>"
# GeoNature source Export id
export_id = 1
# Data type is facultative. By default the value is 'synthese'. Therefore, triggers from to_gnsynthese.sql are not activated.
# If you want to insert your date into a GeoNature database please choose either 'synthese_with_cd_nomenclature' or 'synthese_with_label'.
# If not, delete the line.
data_type = "synthese_with_cd_nomenclature"
[[source]]
# Source configuration
name = "Source2"
user_name = "<monuser>"
user_password = "<monPwd>"
url = "<http://geonature2/>"
export_id = 1
data_type = "synthese_with_cd_nomenclature"
InitDB Schema and tables
To create json tables where datas will be stored, run :
gn2pg_cli --json-tables-create <myconfigfile>
Full download
To download all datas from API, run :
gn2pg_cli --full <myconfigfile>
Incremental download
To update data since last download, run :
gn2pg_cli --update <myconfigfile>
To automate the launching of updates, you can write the cron task using the following command, for example every 30 minutes.
*/30 * * * * /usr/bin/env bash -c "source <path to python environment>/bin/activate && gn2pg_cli --update <myconfigfile>" > /dev/null 2>&1
Debug mode
Debug mode can be activated using --verbose CLI argument
Logs
Log files are stored in $HOME/.gn2pg/log directory.
Import datas into GeoNature datas
Default script to auto populate GeoNature is called “to_gnsynthese”.
gn2pg_cli --custom-script to_gnsynthese <myconfigfile>
Contributing
All devs must be done in forks (see GitHub doc).
Pull requests must be pulled to dev branch.
Install project and development requirements (require poetry):
poetry install
Make your devs and pull requests.
Test gn2pg_cli in dev mode by running this command:
poetry run gn2pg_cli <options>
Renew requirements file for non poetry developers
poetry export -f requirements.txt > requirements.txt
Licence
Team
@lpofredc (LPO Auvergne-Rhône-Alpes), main developper
With the financial support of the DREAL Auvergne-Rhône-Alpes.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for gn2pg_client-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fcaaef2ba84b420b9438f21350affc8b17d8dbf701e3c42cfcbd2fa85f6fe8c4 |
|
MD5 | 358c3b9375009e5edc49bac83ffaee33 |
|
BLAKE2b-256 | 16ab22691864498a2a7e73fcebd3de25d60be4813b0098fca27c861cc08417d6 |