Simple data processing (ETL) library with support for multi-processing
Project description
Python Simple Flow
- Simple framework with a defined structure for data processing with support for multi-process
- Data processing flow is split into four phases:
- Source: Fetch the data from a source
- Ingress: Register the data received, log or in DB for trace-ability
- Transform: Business logic of transforming the data to your business needs
- Egress: Push the data to downstream or your storage
- Processing can be done in one process, like running it in-process or in multi processing mode
- Mode to be chosen depends on the task you are doing
- Easy setting up logger even in multi processing mode
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for py_simple_flow-2020.6.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5653039e8116447012b50cc4599d214f065b731fbf97979ed6753c871b678235 |
|
MD5 | 5ad6aa29c46de769de55aa07f9b995f7 |
|
BLAKE2b-256 | 2e1c023ff91a99d070c934ac2a4e21a5ccbafb4cde6e5e827037db9594219f83 |