Simple data processing (ETL) library with support for multi-processing
Project description
Python Simple Flow
- Simple framework with a defined structure for data processing with support for multi-process
- Data processing flow is split into four phases:
- Source: Fetch the data from a source
- Ingress: Register the data received, log or in DB for trace-ability
- Transform: Business logic of transforming the data to your business needs
- Egress: Push the data to downstream or your storage
- Processing can be done in one process, like running it in-process or in multi processing mode
- Mode to be chosen depends on the task you are doing
- Easy setting up logger even in multi processing mode
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for py-simple-flow-2020.8.23.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b0c3a0978a4af2caf8b86954942e0875b7c1cb49248209f618266655ff7cb553 |
|
MD5 | 40bd5d085d303d60842b75bcf725ed45 |
|
BLAKE2b-256 | ce0791ace51fa19fbe0078a02a47a0b2ccb6df78d27fd4a1e015b41860fb10f5 |
Close
Hashes for py_simple_flow-2020.8.23.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 61597a145850ba162a6a56c7d9eb0cdcaaa912dcae161993b702cb442e086065 |
|
MD5 | 6181300d7e84533008015301b23f5c62 |
|
BLAKE2b-256 | fcdec6d8a49f401caede0f1d9a77e460aea086d8f34142e60b8b116c06657d13 |