Simple data processing (ETL) library with support for multi-processing
Project description
Python Simple Flow
- Simple framework with a defined structure for data processing with support for multi-process
- Data processing flow is split into four phases:
- Source: Fetch the data from a source
- Ingress: Register the data received, log or in DB for trace-ability
- Transform: Business logic of transforming the data to your business needs
- Egress: Push the data to downstream or your storage
- Processing can be done in one process, like running it in-process or in multi processing mode
- Mode to be chosen depends on the task you are doing
- Easy setting up logger even in multi processing mode
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for py-simple-flow-2020.6.7.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1a40810b00bfe6899a9b7a976e0df09d31e6c66c740c3d1f2cecbb81398e3330 |
|
MD5 | 5d96bf068b9f012bd7e82eb67563e2ac |
|
BLAKE2b-256 | 8200cb10864d4ac6c071bb85ab139635cbf036319bea5967ab7da88918789332 |
Close
Hashes for py_simple_flow-2020.6.7.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c0f6b78f85ead8f8d93e66870fd6e057b87a4164454bf0338bdcea8e571d3a17 |
|
MD5 | 1ee39034cf23fe574ade46502c14ece2 |
|
BLAKE2b-256 | 684cf81070810cdddb086620b640ef7453101c324e6821cdebddeff3a15aafc6 |