Skip to main content

Git for data scientists - manage your code and data together

Project description

DVC logo

WebsiteDocsTwitterChat (Community & Support)TutorialMailing List

Travis Windows Build Code Climate Codecov

Data Science Version Control or DVC is an open-source tool for data science and machine learning projects. With a simple and flexible Git-like architecture and interface it helps data scientists:

  1. manage machine learning models - versioning, including data sets and transformations (scripts) that were used to generate models;

  2. make projects reproducible;

  3. make projects shareable;

  4. manage experiments with branching and metrics tracking;

It aims to replace tools like Excel and Docs that are being commonly used as a knowledge repo and a ledger for the team, ad-hoc scripts to track and move deploy different model versions, ad-hoc data file suffixes and prefixes.

How DVC works

DVC is compatible with Git for storing code and the dependency graph (DAG), but not data files cache. To store and share data files cache DVC supports remotes - any cloud (S3, Azure, Google Cloud, etc) or any on-premise network storage (via SSH, for example).

how_dvc_works

Quick start

Please read Get Started for the full version. Common workflow commands include:

Step

Command

Track code and data together

$ git add train.py
$ dvc add images.zip

Connect code and data by commands

$ dvc run -d images.zip -o images/ unzip -q images.zip
$ dvc run -d images/ -d train.py -o model.p python train.py

Make changes and reproduce

$ vi train.py
$ dvc repro model.p.dvc

Share code

$ git add .
$ git commit -m 'The baseline model'
$ git push

Share data and ML models

$ dvc remote add myremote s3://mybucket/image_cnn
$ dvc config core.remote myremote
$ dvc push

Installation

There are three options to install DVC: pip, Homebrew, or an OS-specific package:

pip (PyPI)

Stable

pip install dvc

Development

pip install git+git://github.com/iterative/dvc

Homebrew

brew install iterative/homebrew-dvc/dvc

or:

brew cask install iterative/homebrew-dvc/dvc

Package

Self-contained packages for Windows, Linux, Mac are available. The latest version of the packages can be found at GitHub releases page.

Ubuntu / Debian (apt)

sudo wget https://dvc.org/deb/dvc.list -O /etc/apt/sources.list.d/dvc.list
sudo apt-get update
sudo apt-get install dvc

Fedora / CentOS (rpm)

sudo wget https://dvc.org/rpm/dvc.repo -O /etc/yum.repos.d/dvc.repo
sudo yum update
sudo yum install dvc

Arch linux (AUR)

Unofficial package, any inquiries regarding the AUR package, refer to the maintainer.

yay -S dvc

Contributing

Contributions are welcome! Please see our Contributing Guide for more details.

0 1 2 3 4 5 6 7

Mailing List

Want to stay up to date? Want to help improve DVC by participating in our occasional polls? Subscribe to our mailing list. No spam, really low traffic.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dvc-0.28.2.tar.gz (87.7 kB view hashes)

Uploaded Source

Built Distribution

dvc-0.28.2-py2.py3-none-any.whl (137.1 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page