4 projects
pydiverse-transform
Pipe based dataframe manipulation library that can also transform data on SQL databases
pydiverse-pipedag
A pipeline orchestration library executing tasks within one python session. It takes care of SQL table (de)materialization, caching and cache invalidation. Blob storage is supported as well for example for storing model files.
pydiverse.pipetest
An adaption layer for pydiverse.pipedag that simplyfies execution of pipedag steps as unit tests with cache invalidation awareness.
pydiverse
A collection of nicely interoperable libraries for data pipeline orchestration allowing for both SQL target and in-memory operation.
