visin

Send training runs to Visin: epochs, test results, benchmarks, prediction frames and configs. Read them back for analysis.

pip install visin

import visin

with visin.init("unet baseline", project="road-seg") as run:
    for epoch in range(1, 21):
        train_loss, val_loss, val_iou = train_one_epoch()
        run.log_epoch(epoch, train={"loss": train_loss}, val={"loss": val_loss, "mean_iou": val_iou})

Point it at your Visin with two environment variables. Without them, the same script runs and reports nothing.

export VISIN_URL=https://vision-api.visin.eu
export VISIN_TOKEN=…            # a pipeline key, from the project's Settings → Pipeline keys
visin check --write             # confirm this machine can report

What it does for you

• Never stops your training. Reports go out from a background thread, and failures are logged, never raised.
• Retries safely. Epochs and test results carry their own ids, so a retried request is never stored twice.
• Survives the network. Reports wait on disk while Visin is unreachable, and are sent when it answers. With VISIN_MODE=offline, a compute node with no route to Visin keeps everything for visin sync.
• Keeps crashed runs. A run that crashes, or that the scheduler terminates, still sends its epochs and is marked failed.
• Takes metrics as they are. NumPy and PyTorch scalars are converted, and NaN becomes a gap in the chart.
• Reports once in distributed jobs. Only rank zero reports.

More than epochs

run.log_test_results(12, {"day": {"car": {"iou": 0.74}}, "night": {"car": {"iou": 0.61}}})
run.log_benchmark({"device": "cuda", "batch_size": 1, "fps": 119.0})  # machine details filled in
run.upload_visualization(12, "renders/overlay_0001.png", kind="overlay")
run.log_config(args)  # argparse, dataclass, pydantic, Hydra…

Rank checkpoints

checkpoint = visin.local_checkpoint("checkpoints/epoch_40.pth")
evaluation = visin.evaluate(
    results,
    suite="road-test@1",
    checkpoint=checkpoint,
    sample_counts={"day": 1200, "night": 800},
    project="road-seg",
)
print(evaluation.verdict)  # eligible, or incomplete: missing-condition(night)

Visin judges each result against a written-down suite and says whether it can be ranked, and why not. visin leaderboard road-test@1 shows the ranking and where each model is weakest. See evaluating models.

Frameworks

from visin.integrations.lightning import VisinCallback  # or visin.integrations.keras

trainer = L.Trainer(callbacks=[VisinCallback(name="segformer b2", project="road-seg")])

Reading runs back

api = visin.Api()
for run in api.trainings(project="road-seg", status="completed"):
    frame = api.epochs_frame(run)  # pip install 'visin[pandas]'

Datasets

Public datasets on Visin need no account or token. They can be listed and downloaded, once, into VISIN_DATA_DIR:

export VISIN_DATASET_URL=https://dataset-api.visin.eu
visin datasets
visin download zod     # resumes if interrupted; prints the folder

visin.Datasets does the same from Python.

Documentation

visin-platform.github.io/visin-py has the getting-started guide, guides to offline use and each framework, and the API reference.

Contributing

See CONTRIBUTING.md. In short: make install, then make check before a pull request. Changes are listed in the changelog.

License

MIT