Skip to main content

Transformer Lab SDK

The Transformer Lab Python SDK provides a way for ML scripts to integrate with Transformer Lab.

Install

pip install transformerlab-internal

Usage

from lab import lab

# Initialize with experiment ID
lab.init("my-experiment")
lab.log("Job initiated")

config_artifact_path = lab.save_artifact(<config_file>, "training_config.json")
lab.log(f"Saved training config: {config_artifact_path}")
lab.update_progress(1)

...
lab.update_progress(99)

model_path = lab.save_model(<training_output_dir>, name="trained_model")
lab.log("Saved model file to {model_path}")

lab.finish("Training completed successfully")

Shared storage

Each user has a personal area of the team workspace for large files reused across experiments and jobs — a base model, a tokenized corpus, a reference checkpoint. Upload once from the terminal with lab storage upload, then read it from any job:

from lab import lab

corpus = lab.storage_download("tokenized-corpus")                 # directory → local dir
weights = lab.storage_download("models/base-model.safetensors")   # file → local path

for entry in lab.storage_list("models"):
    print(entry["relpath"], entry["size"])

lab.storage_upload("./derived-index.faiss", "indexes/wiki.faiss")

Downloads always land on the node's own disk — the returned path is a file you can open. By default they go to a cache under ~/.transformerlab/cache/user_storage/ (pass dest= to choose a location) and read straight from object storage, so a multi-GB file moves at full bandwidth without routing through the API server. A local copy whose size already matches is reused, so re-running a job does not re-fetch it; pass force=True to override.

Every path is relative to your own storage root. Async variants exist for all three (async_storage_download, async_storage_upload, async_storage_list).

Reporting metrics

Pass a dict of named metrics to lab.finish(score=...). The dict shape is required — metrics are stored under job_data.score and surfaced by lab job list (Score column) and lab job info, and read by sweep / autoresearch flows for optimization.

lab.finish(message="Done!")                              # success, no score
lab.finish(message="Done!", score={"accuracy": 0.78})    # success with one metric
lab.finish(score={"accuracy": 0.78, "f1": 0.83})         # multiple metrics

Do not pass a scalar (e.g. lab.finish(score=0.78)) — wrap it in a dict instead: lab.finish(score={"score": 0.78}).

Sample scripts can be found at https://github.com/transformerlab/transformerlab-platform/tree/main/lab-sdk/scripts/examples

Development

The code for this can be found in the lab-sdk directory of https://github.com/transformerlab/transformerlab-platform

To develop locally in editable mode and run automated tests:

cd lab-sdk
uv venv
uv pip install -e .
uv run pytest  # Run tests

Metadata

Release files for transformerlab-internal 0.1.64

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for transformerlab-internal 0.1.64
File Size Uploaded Provenance Actions
transformerlab_internal-0.1.64.tar.gz 115.8 kB Without provenance Details for transformerlab_internal-0.1.64.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for transformerlab-internal 0.1.64
File Interpreter ABI Platform Actions
transformerlab_internal-0.1.64-py3-none-any.whl Python 3 none any Details for transformerlab_internal-0.1.64-py3-none-any.whl

Total release size: 207.6 kB

Release files / transformerlab_internal-0.1.64.tar.gz

Download URL transformerlab_internal-0.1.64.tar.gz
Size 115.8 kB
Tags Source
SHA-256 checksum
How to use checksums
210f2a59087367e1748c5f7fdda19e2ea8e8ae67077af628d7823fb09833620e
BLAKE2b-256 checksum
How to use checksums
a0db54404c5566ee7e20d919b106b16772c632e069fe69f050496f457cd3a217
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / transformerlab_internal-0.1.64-py3-none-any.whl

Download URL transformerlab_internal-0.1.64-py3-none-any.whl
Size 91.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
faf9c90515b3553e4fececba5c458881e059719b66c6583a6c2d4f851e8579c9
BLAKE2b-256 checksum
How to use checksums
1d60c7ee359b0542758322e4991ce7ec2b2f625b27e16aa6082940b9c832eb82
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for transformerlab-internal 0.1.64

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page