DAGpy
DAGpy is a data science collaboration tool based on iPython notebooks enabling data science teams to:
easily collaborate by branching out of others’ notebooks
minimize code duplication
give a clean overview of the project
cache intermediate outputs so team members can use them without re-evaluation
automate the process of code execution upon data changes or on schedule
provide a clean interface to the data visualization dashboard designers and developers
DAGpy manages a DAG (directed acyclic graph) of blocks of code, with each block being a sequence of iPython notebook cells, together with their outputs. It is designed to work seamlessly with popular VC systems like git and can be run locally or as a server application.
GitHub: github.com/ibestvina/dagpy/.
Author: Ivan Bestvina
Example project
To play around with the example project, you can:
view the project DAG: dagpy view
run all the blocks: dagpy execute -a
add blocks through flows (with block B as a parent) and run them automatically: dagpy makeflow B -r
commit the changes: dagpy submitflow dagpy_flow.ipynb
explore other DAGpy options with dagpy -h
Please note that notebook execution time includes a significant overhead of over a second, because a kernel must be started for each one. In future, we plan on adding support for non-notebook plane python blocks. These would also be edited through a flow notebook view, but would be saved as .py scripts, and executed without noticable overhead.
Dependencies:
python 3
jupyter
dill
networkx, matplotlib (for DAG view)
1 maintainer
Metadata
1 maintainer
Release files for dagpy 0.3.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| dagpy-0.3.2.zip | 17.5 kB | Without provenance | Details for dagpy-0.3.2.zip |
Release files / dagpy-0.3.2.zip
| Download URL | dagpy-0.3.2.zip |
|---|---|
| Size | 17.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f22aadafb3bfb9ff4f296b3a483106e9d41458ab641b4837a8f0a42e24ee9925
|
|
BLAKE2b-256 checksum How to use checksums |
af6d8f574629b84bddc4b8ca993fed71fa157efcb1c95e3716fd259a32667d87
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for dagpy 0.3.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
