Skip to main content

Python API to Biolovision API and applications using it

Project description

https://img.shields.io/badge/code%20style-black-000000.svg PyPI - Status PyPI - Python Version PyPI - License

Description

Python applications that use Biolovision/VisioNature (VN) API to:

  • download data from VN sites and stores it to a Postgresql database.
  • update sightings directly in VN site

Applications are available either as:

  • Python modules from PyPI
  • Docker images from Docker Hub

They are tested under Linux Ubuntu or Debian. Other Linux distributions could work. Windows is not tested at all and will probably not work.

Installation - Python

These instructions present the steps required to install the Python applications.

Create a python virtual environment, activate it and update basic tools:

python3 -m venv VN_env
source VN_env/bin/activate
python -m pip install --upgrade pip

Install from PyPI:

pip install Client-API-VN

Installation - Docker

These instructions present the steps required to install the Docker applications:

docker pull dthonon/client-api-vn
docker run --name xfer_vn \
           --mount source=xfer_vn,target=/root \
           --workdir /root \
           --tty --interactive \
           dthonon/client-api-vn bash

The following steps are the common to both Python and Docker installation.

Getting Started - transfer_vn

Initialize the sample YAML file in your HOME directory and edit with your local details. The YAML file is self documented:

transfer_vn --init .evn_your_site.yaml
editor $HOME/.evn_your_site.yaml

Create the database and tables:

transfer_vn --db_create --json_tables_create --col_tables_create .evn_your_site.yaml

You can then download data, as enabled and filtered in the YAML file. Beware that, depending on the volume of observations, this can take several hours. We recommend starting with a small taxonomic group first:

transfer_vn --full .evn_your_site.yaml

After this full download, data can be updated. For observations, only new, modified or deleted observations are downloaded. For other controlers, a full download is always performed. Each controler runs on its own schedule, defined in the YAML configuration file. To create or update, after modifying the configuration file, the schedule:

transfer_vn --schedule .evn_your_site.yaml

Once this is done, you can update the database with new observations:

transfer_vn --update .evn_your_site.yaml

Note: this script should run hourly or dayly in a cron job. It must run at least every week.

Getting Started - update_vn

Initialize the sample YAML file in your HOME directory and edit with your local details. The YAML file is self documented:

update_vn --init .evn_your_site.yaml
editor $HOME/.evn_your_site.yaml

Prerequisites

For Linux and Postgresql installation, refer to server installation

Installation requires the following python module:

pip

All other python dependencies are managed by pip install.

Command-line options - transfer_vn

The application runs as:

transfer_vn options config

where:

  • options command line options described below
  • config YAML file, located in $HOME directory, described in sample file
-h, --help Prints help and exits
--version Print version number
--verbose Increase output verbosity
--quiet Reduce output verbosity
--init Initialize the YAML configuration file
--db_drop Delete if exists database and roles
--db_create Create database and roles
--json_tables_create
 Create or recreate json tables
--col_tables_create
 Create or recreate colums based tables
--full Perform a full download
--update Perform an incremental download
--schedule Create or update the incremental update schedule
--status Print downloading status (schedule, errors…)
--count Count observations by site and taxo_group
--profile Gather and print profiling times

Command-line options - update_vn

The application runs as:

update_vn options config input

where:

  • options command line options described below
  • config YAML file, located in $HOME directory, described in sample file
  • input CSV file listing sightings to be updated
-h, --help Prints help and exits
--version Print version number
--verbose Increase output verbosity
--quiet Reduce output verbosity
--init Initialize the YAML configuration file

CSV input file must contain the following columns:

  • site, as defined in YAML site section
  • id_universal of the sighting to modify
  • path to the attribute to modify, in JSONPath syntax
  • operation: - replace: add if not present or update a sighting attribute
  • value: new value inserted or updated

For example:

site;id_universal;path;operation;value
Isère;2246086;$['data']['sightings'][0]['observers'][0]['atlas_code'];replace;4

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for Client-API-VN, version 2.6.4
Filename, size File type Python version Upload date Hashes
Filename, size Client_API_VN-2.6.4-py2.py3-none-any.whl (76.6 kB) File type Wheel Python version py2.py3 Upload date Hashes View
Filename, size Client_API_VN-2.6.4.tar.gz (137.9 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page