Skip to main content

Experiments logger for ML projects.

Project description

DVCLive

PyPI Status Python Version License

Tests Codecov pre-commit Black

DVCLive is a Python library for logging machine learning metrics and other metadata in simple file formats, which is fully compatible with DVC.

Documentation


Quickstart

Python API Overview PyTorch Lightning Scikit-learn Ultralytics YOLO v8

Install dvclive

$ pip install dvclive

Initialize DVC Repository

$ git init
$ dvc init
$ git commit -m "DVC init"

Example code

Copy the snippet below into train.py for a basic API usage example:

import time
import random

from dvclive import Live

params = {"learning_rate": 0.002, "optimizer": "Adam", "epochs": 20}

with Live() as live:

    # log a parameters
    for param in params:
        live.log_param(param, params[param])

    # simulate training
    offset = random.uniform(0.2, 0.1)
    for epoch in range(1, params["epochs"]):
        fuzz = random.uniform(0.01, 0.1)
        accuracy = 1 - (2 ** - epoch) - fuzz - offset
        loss = (2 ** - epoch) + fuzz + offset

        # log metrics to studio
        live.log_metric("accuracy", accuracy)
        live.log_metric("loss", loss)
        live.next_step()
        time.sleep(0.2)

See Integrations for examples using DVCLive alongside different ML Frameworks.

Running

Run this a couple of times to simulate multiple experiments:

$ python train.py
$ python train.py
$ python train.py
...

Comparing

DVCLive outputs can be rendered in different ways:

DVC CLI

You can use dvc exp show and dvc plots to compare and visualize metrics, parameters and plots across experiments:

$ dvc exp show
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
Experiment                 Created    train.accuracy   train.loss   val.accuracy   val.loss   step   epochs
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
workspace                  -                  6.0109      0.23311          6.062    0.24321      6   7
master                     08:50 PM                -            -              -          -      -   -
├── 4475845 [aulic-chiv]   08:56 PM           6.0109      0.23311          6.062    0.24321      6   7
├── 7d4cef7 [yarer-tods]   08:56 PM           4.8551      0.82012         4.5555   0.033533      4   5
└── d503f8e [curst-chad]   08:56 PM           4.9768     0.070585         4.0773    0.46639      4   5
─────────────────────────────────────────────────────────────────────────────────────────────────────────────
$ dvc plots diff $(dvc exp list --names-only) --open

dvc plots diff

DVC Extension for VS Code

Inside the DVC Extension for VS Code, you can compare and visualize results using the Experiments and Plots views:

VSCode Experiments

VSCode Plots

While experiments are running, live updates will be displayed in both views.

DVC Studio

If you push the results to DVC Studio, you can compare experiments against the entire repo history:

Studio Compare

You can enable Studio Live Experiments to see live updates while experiments are running.


Comparison to related technologies

DVCLive is an ML Logger, similar to:

The main differences with those ML Loggers are:

  • DVCLive does not require any additional services or servers to run.
  • DVCLive metrics, parameters, and plots are stored as plain text files that can be versioned by tools like Git or tracked as pointers to files in DVC storage.
  • DVCLive can save experiments or runs as hidden Git commits.

You can then use different options to visualize the metrics, parameters, and plots across experiments.


Contributing

Contributions are very welcome. To learn more, see the Contributor Guide.

License

Distributed under the terms of the Apache 2.0 license, dvclive is free and open source software.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dvclive-3.48.0.tar.gz (2.0 MB view details)

Uploaded Source

Built Distribution

dvclive-3.48.0-py3-none-any.whl (43.5 kB view details)

Uploaded Python 3

File details

Details for the file dvclive-3.48.0.tar.gz.

File metadata

  • Download URL: dvclive-3.48.0.tar.gz
  • Upload date:
  • Size: 2.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.0 CPython/3.12.4

File hashes

Hashes for dvclive-3.48.0.tar.gz
Algorithm Hash digest
SHA256 42434660f16b88c8931e625da010c9cdabf2b0d8d11173b81ed1d976c4b7fb0a
MD5 85e39ede60469eb3c7a968b040de3e3e
BLAKE2b-256 e391d6cf2ccceaa063b988809ab4fa33846e6ee811ef146b573dadc1c9fb0a0f

See more details on using hashes here.

File details

Details for the file dvclive-3.48.0-py3-none-any.whl.

File metadata

  • Download URL: dvclive-3.48.0-py3-none-any.whl
  • Upload date:
  • Size: 43.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.0 CPython/3.12.4

File hashes

Hashes for dvclive-3.48.0-py3-none-any.whl
Algorithm Hash digest
SHA256 6625b9eac49322165877a95696db8934d190d89fadf53294c471d2584b322879
MD5 702beeffe609317b257dd4f38a805c79
BLAKE2b-256 df8696f6a2383e94b90b97b7920c4a1831a8858a5bfd1a08ac3bd8e2626e7bd1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page