A Python package for parameter and data version control with DVC
Project description
|build| |docs| |license| |code style|
.. image:: https://raw.githubusercontent.com/zincware/ZnTrack/main/docs/source/img/zntrack.png
Parameter Tracking for Python
ZnTrack [zɪŋk træk] is an easy-to-use package for tracking parameters in your Python projects. What is a parameter? Anything set by a user in your code, for example, the number of layers in a neural network layer or the window size of a moving average. ZnTrack works by storing the values of parameters in Python classes and functions and monitoring how they change for several different runs. These changes can then be compared graphically to see what effect they had on your workflow. Beyond the standard tracking of parameters in a project, ZnTrack can be used to deploy jobs with a set of different parameter values, avoid the re-running of components of code where parameters have not changed, and to identify computational bottlenecks in your code.
Example
With ZnTrack a DVC Node on the computational graph can be written as a Python class. DVC Options, such as parameters, input dependencies and output files are class attributes.
.. code-block:: py
from zntrack import Node, dvc
from random import randrange
@Node()
class HelloWorld:
"""Define a ZnTrack Node"""
# parameter to be tracked
max_number = dvc.params()
# parameter to store as output
random_number = dvc.result()
def __call__(self, max_number):
"""Pass tracked arguments"""
self.max_number = max_number
def run(self):
"""Command to be run by DVC"""
self.random_number = randrange(self.max_number)
This stage can be used via
.. code-block:: py
hello_world = HelloWorld()
hello_world(max_number=512)
which builds the DVC stage and can be used e.g., through :code:dvc repro
.
The results can then be accessed easily via :code:HelloWorld(load=True).random_number
.
More detailed examples and further information can be found in the ZnTrack Documentation <https://zntrack.readthedocs.io/en/latest/>
_.
Technical Details
ZnTrack as an Object-Relational Mapping for DVC
On a fundamental level the ZnTrack package provides an easy-to-use interface for DVC directly from Python.
It handles all the computational overhead of reading config files, defining outputs in the dvc.yaml
as well as in the script and much more.
For more information on DVC visit their homepage <https://dvc.org/doc>
_.
Installation
Simply run:
.. code-block:: bash
pip install zntrack
Or you can install from source with:
.. code-block:: bash
git clone https://github.com/zincware/ZnTrack.git cd ZnTrack pip install .
.. badges
.. |build| image:: https://github.com/zincware/ZnTrack/actions/workflows/pytest.yaml/badge.svg :alt: Build tests passing :target: https://github.com/zincware/py-test/blob/readme_badges/
.. |docs| image:: https://readthedocs.org/projects/zntrack/badge/?version=latest :target: https://zntrack.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status
.. |license| image:: https://img.shields.io/badge/License-EPL-purple.svg?style=flat :alt: Project license :target: https://www.eclipse.org/legal/epl-2.0/faq.php
.. |code style| image:: https://img.shields.io/badge/code%20style-black-black :alt: Code style: black :target: https://github.com/psf/black/
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.