dpetl — Data package ETL
The dpetl is a command-line interface (CLI) tool designed to run the three ETL phases (Extract, Transform, Load).
It is designed to work alongside the Data Package standard specification.
Installation
It requires Python 3.10 or more.
# Install:
pip install dpetl
poetry add dpetl
# Optional dependencies
pip install dpetl[github-app]
poetry install --extras github-app
Usage
Use dpetl --help to inspect the CLI.
dpetl extract # download the data (modes: api, email, cli)
dpetl transform # rename fields, anonymize values and export the files
dpetl load # publish the data to a GitHub repository
How it works
The CLI reads the Data Package descriptor:
-
datapackage.yamlwhen runningextractortransform -
datapackage.jsonwhen runningload
Then it runs each phase on its resources, following the dpetl_extract, dpetl_transform and dpetl_load properties declared in it.
If you have more than one data package, place each one in its own subdirectory of datapackages/.
Documentation
Full documentation, with a page for each phase, the descriptor details and the API reference, at https://splor-mg.github.io/dpetl/.
2 maintainers
Metadata
2 maintainers
Release files for dpetl 0.18.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| dpetl-0.18.1.tar.gz | 16.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dpetl-0.18.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 38.3 kB
Release files / dpetl-0.18.1.tar.gz
| Download URL | dpetl-0.18.1.tar.gz |
|---|---|
| Size | 16.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
17b003520bad0097b814de2098b70a4629660c3edc1de770049f8d6f3436a791
|
|
BLAKE2b-256 checksum How to use checksums |
5dca43200539238c051e7ee3ed4dcb2f746e6b527487a2d1c38265dd05d6c65b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.5.1 CPython/3.13.15 Linux/6.17.0-1022-azure
|
Release files / dpetl-0.18.1-py3-none-any.whl
| Download URL | dpetl-0.18.1-py3-none-any.whl |
|---|---|
| Size | 22.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3e58e71a83e7885328f531b4b549a1bc15d2a54fe993028d96eb82b37b153e40
|
|
BLAKE2b-256 checksum How to use checksums |
209db0c7a258a9072b063d15b6f9e58aaef63bd6f5061879d9c1d853dec07569
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.5.1 CPython/3.13.15 Linux/6.17.0-1022-azure
|
