CacheFlow
CacheFlow is a caching workflow engine, capable of executing dataflows while reusing previous results where appropriate, for efficiency. It is very extensible and can be used in many projects.
Goals
☑ Python 3 workflow system
☑ Executes dataflows from JSON or YAML files
☑ Extensible: can add new components, new storage formats, new caching mechanism, new executors
☐ Pluggable: extensions can be installed from PyPI without forking
☑ Re-usable: can execute workflows by itself, but can also be embedded into applications. Some I plan on developing myself:
☑ Literate programming app: snippets or components embedded into a markdown file, which are executed on render (similar to Rmarkdown). Results would be cached, making later rendering fast
☐ Integrate in some of my NYU research projects (VisTrails, Vizier, D3M)
Other ideas:
☐ Use Jupyter kernels as backends to execute code (giving me quick access to all the languages they support)
☐ Isolate script execution (to run untrusted Python/… code, for example with Docker)
Non-goals
Make a super-scalable and fast workflow execution engine: I’d rather make executors based on Spark, Dask, Ray than try to re-implement those from scratch.
Status
Basic structures are here, extracted from D3M. Execution works. Very few components available. Working on web interface.
1 maintainer
Metadata
1 maintainer
Release files for cacheflow 0.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| cacheflow-0.3.tar.gz | 232.4 kB | Without provenance | Details for cacheflow-0.3.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| cacheflow-0.3-py3-none-any.whl | Python 3 | none | any | Details for cacheflow-0.3-py3-none-any.whl |
Total release size: 472.4 kB
Release files / cacheflow-0.3.tar.gz
| Download URL | cacheflow-0.3.tar.gz |
|---|---|
| Size | 232.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3e9aabef06ae7cbfceb25f3a9e0b672a3ba18a620f8cf957513ddc14caeb0702
|
|
BLAKE2b-256 checksum How to use checksums |
2d02293a2299acd5a4f63188dda52ce341ebbd1f22b4c6e70b28414d225bbb07
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/41.6.0 requests-toolbelt/0.8.0 tqdm/4.19.9 CPython/3.6.3
|
Release files / cacheflow-0.3-py3-none-any.whl
| Download URL | cacheflow-0.3-py3-none-any.whl |
|---|---|
| Size | 240.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
552569e6207558c132c2fe26eb18fb7b529ef9b0d9622cca7921f2a4af3f1830
|
|
BLAKE2b-256 checksum How to use checksums |
11188677bb2c3b3ca37c286b22abbe61cd0f65fc3f6831f01709670b70082536
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.4.2 requests/2.22.0 setuptools/41.6.0 requests-toolbelt/0.8.0 tqdm/4.19.9 CPython/3.6.3
|
Security for cacheflow 0.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
