dbnomics-fetcher-ops 0.6.2

Manage DBnomics fetchers

DBnomics fetcher ops

Manage DBnomics fetchers: list, configure and run pipelines.

Install

pip install dbnomics-fetcher-ops

Usage

Configure a fetcher

The configure command needs write privileges. Create a GitLab personal access token having the api scope, and pass it using the --gitlab-private-token option or the GITLAB_PRIVATE_TOKEN environment variable in ~/.config/dbnomics/dbnomics-fetchers.env.

dbnomics-fetchers -v configure scsmich --dry-run
# If everything seems OK, remove the --dry-run flag:
dbnomics-fetchers -v configure scsmich

List fetchers

dbnomics-fetchers -v list

Run fetcher pipelines

# Replace PROVIDER_SLUG by the real value:
dbnomics-fetchers -v run --provider-slug PROVIDER_SLUG

# To run a pipeline for each fetcher:
dbnomics-fetchers -v list --slug | xargs -I {} dbnomics-fetchers -v run --provider-slug {}

Development

Using uv is highly recommended.

git clone https://git.nomics.world/dbnomics/dbnomics-fetcher-ops.git
cd dbnomics-fetcher-ops
uv sync
. .venv/bin/activate
cp .env.example .env

Use with:

dbnomics-fetchers COMMAND