Python function to ingest data from UN Comtrade's bulk API to PostgreSQL
Project description
pg-comtrade
Python utility function to ingest data from UN Comtrade's bulk API into a (partitioned) PostgreSQL table.
[!IMPORTANT] This is a work-in-progress. This README serves as a rough design spec.
Installation
pg-comtrade can be installed from PyPI using pip. psycopg2 or psycopg (Psycopg 3) must also be explicitly installed.
pip install pg-comtrade psycopg
Usage
from pg_comtrade import sync_goods_yearly
# For example purposes, PostgreSQL can be run locally using this...
# docker run --rm -it -e POSTGRES_HOST_AUTH_METHOD=trust -p 5432:5432 postgres
# ... which should work with this engine
engine = sa.create_engine('postgresql+psycopg://postgres@127.0.0.1:5432/')
# ... to ingest into the un.comtrade_goods table
with engine.connect() as conn:
sync_goods_yearly(conn, 'un', 'comtrade_goods', subscription_key='123456abcdef')
Development / running tests
To develop pg-comtrade, you will need to install the package in editable mode, and install development dependencies.
pip install -e '.[dev]'
And start a local PostgreSQL server:
./start-services.sh
And then to run the tests:
pytest
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pg_comtrade-0.0.2.tar.gz
(4.4 kB
view hashes)
Built Distribution
Close
Hashes for pg_comtrade-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f8a64223366a4f01360b9d190ba0911e5ca720ff15ac41f3948dfed5edf687f4 |
|
MD5 | e221b523152c8d767ca7d917a15e52b0 |
|
BLAKE2b-256 | c94b879a30d655b2ddfce61b50c105805cda2f93c0ecc9983f65e8173f35ac83 |