neptune-scale 0.30.0

A minimal client library

neptune.ai

Quickstart   •   Website   •   Docs   •   Examples   •   Blog  

---

Neptune is an experiment tracker purpose-built for foundation model training.

With Neptune, you can monitor thousands of per-layer metrics—losses, gradients, and activations—at any scale. Visualize them with no lag and no missed spikes. Drill down into logs and debug training issues fast. Keep your model training stable while reducing wasted GPU cycles.

Watch a 3min explainer video →  

Play with a live example project in the Neptune app →  

Get started

Neptune consists of a Python API and a web application.

Install the Python client library:

pip install neptune-scale

Configure the API token and project:

1. Log in to your Neptune workspace.

2. Get your API token from your user menu in the bottom left corner.

   To create shared API tokens, see how to set up a service account.

3. In the environment where neptune-scale is installed, save your API token to the NEPTUNE_API_TOKEN environment variable:

   export NEPTUNE_API_TOKEN="h0dHBzOi8aHR0cHM6...Y2MifQ=="
   

4. Create a project, or find an existing project you want to send the run metadata to.

   To create a project via API:

   from neptune_scale.projects import create_project
   
   create_project(
       name="project-x",
       workspace="team-alpha",
   )
   

5. (optional) In the environment where neptune-scale is installed, save your full project path to the NEPTUNE_PROJECT environment variable:

   export NEPTUNE_PROJECT="team-alpha/project-x"
   

   If you skip this step, you need to pass the project name as an argument each time you start a run.

You're ready to start using Neptune.

For more help with setup, see Get started in the Neptune documentation.

Example usage

Create an experiment:

from neptune_scale import Run


run = Run(experiment_name="MyExperimentName")

Then, call logging methods on the run and pass the metadata as a dictionary.

Log configuration or other simple values with log_configs():

run.log_configs(
    {
        "learning_rate": 0.001,
        "batch_size": 64,
    }
)

Inside a training loop or other iteration, use log_metrics() to append metric values:

# inside a loop
for step in range(100):
    run.log_metrics(
        data={"acc": 0.89, "loss": 0.17},
        step=step,
    )

Upload files:

run.assign_files(
    {
        "dataset/data_sample": "sample_data.csv",
        "dataset/image_sample": "input/images/img1.png",
    }
)

Log series of histograms:

from neptune_scale.types import Histogram


for step in epoch:
    # your training loop

    my_histogram = Histogram(...)

    run.log_histograms(
        histograms={
            "layers/1/activations": my_histogram
        },
        step=step,
    )

To help organize and group runs, apply tags:

run.add_tags(tags=["testing", "data v1.0"])

# To assign the run to groups, apply group tags
run.add_tags(
    tags=["group1", "group2"]
    group_tags=True,
)

When logging is no longer needed, use close() to stop the run:

run.close()

To explore your experiment, open the project in Neptune and navigate to Runs. For an example, see the demo project →

For more instructions, see the Neptune documentation:

• Quickstart
• Create a run
• Log metadata
• API reference

Getting help

For help and support, visit our Support Center.

 

---

People behind Neptune

Created with :heart: by the neptune.ai team →