LearnML Python SDK 0.1.4

HTTP(S) URLs select the website API; bare host:port addresses retain direct gRPC.

from getpass import getpass
from learnml import LearnMLClient

with LearnMLClient("https://your-learnml-server.example") as client:
    client.login("you@example.com", getpass("LearnML password: "))
    universe = next(u for u in client.list_universes() if u["name"] == "My experiments")
    uid = universe["id"]
    run = client.create_training_run(uid, "my-experiment", model_name="my-model")
    client.log_metrics(uid, run["id"], step=1, loss=0.42, accuracy=0.91)
    client.upload_data(uid, "experiment-data", "CUSTOM", "experiment.csv")
    client.end_training_run(uid, run["id"])

Install with python -m pip install --upgrade learnml-sdk, or install a local checkout with python -m pip install ./sdk. Restart a notebook kernel after upgrading an already-imported package. HTTP support requires version 0.1.3 or later.

HTTP mode preserves the existing public methods and camelCase response dictionaries. token and refresh state work in both modes. Requests have connection/read timeouts; API and network errors use the SDK exception classes. HTTP multipart uploads stream from the client file; the current gateway still buffers uploads in server memory. chunk_size controls gRPC chunks; the HTTP transport controls its own multipart read sizes. HTTP collection batching uses paginated API calls.

The existing gateway cannot accept custom checkpoint metadata; HTTP save_checkpoint(metadata=...) raises an explicit error if nonempty metadata is supplied. Empty metadata and the normal checkpoint upload/download flow are supported.

Run regression tests with python -m unittest discover -s sdk/tests from the repository root after installing the SDK.

API tokens for Colab and long-running jobs

Sign in to the website, open API Tokens, and select Generate token. Give the token a name and choose No expiry — until revoked, or a fixed expiry. Copy the full token immediately: it is shown only once. Token hashes, names, prefixes and timestamps are stored on the server; full secrets cannot be retrieved later.

Save it in Colab Secrets as LEARNML_API_TOKEN, enable notebook access, and use:

from google.colab import userdata
from learnml import LearnMLClient

client = LearnMLClient(
    "your-learnml-server.example:50051",
    token=userdata.get("LEARNML_API_TOKEN"),
)
# No login or token-refresh loop is required.
print(client.list_universes())

HTTP clients also accept the same token. The token= argument works in older SDK versions; SDK 0.1.4 adds the explicit api_token= alias. Pass one of these arguments, not both.

To manage tokens from SDK 0.1.4, first sign in using client.login(...), then call:

• create_api_token(name, expires_in_days=0) returns { "apiToken": {...}, "token": "lml_..." } once. Zero days means no automatic expiry; 1–3650 days sets an expiry.
• list_api_tokens() returns metadata and token prefixes, never full secrets.
• revoke_api_token(token_id) disables an owned token on its next request.

API tokens inherit your current workspace permissions; removing membership removes access. They cannot create, list, or revoke credentials. Use a password-based login session to manage tokens. Store tokens like passwords; their presence does not add encryption to a plaintext HTTP/gRPC connection.

Create and edit individual rows

For bulk creation, batch_create_data_points(universe_id, rows, collection_id=None) accepts up to 10,000 dictionaries with the same keyword names as create_data_point: name, data_type, labels, metadata, raw_content (bytes), and llm_data (dict). It returns dataPointIds in input order and createdCount. Rows and optional collection membership commit together. Requests are limited to 32 MiB total and 1 MiB per row. Split larger datasets into multiple calls; there are no automatic retries or deduplication after ambiguous network failures. See the repository's docs/row-api.md for batch examples.

Row content can live directly in PostgreSQL, without a bucket file. Inline content is limited to 1 MiB per create/update request. File uploads remain available for larger data. These methods require a server with the row-content update enabled.

row = client.create_data_point(
    universe_id, "sample-001", "LLM_SFT",
    llm_data={"instruction": "Tag names", "input": "Hello Alice",
              "output": "Hello <NAME>Alice</NAME>",
              "metadata": {"spans": '[{"start": 6, "end": 11}]'}},
    labels={"split": "train"},
    metadata={"source": "manual"},
)
row = client.update_data_point(
    universe_id, row["id"],
    llm_data={"input": "Hello Bob", "output": "Hello <NAME>Bob</NAME>"},
)
collection = client.create_collection(universe_id, "Examples")
client.add_to_collection(universe_id, collection["id"], [row["id"]])
row = client.get_data_point(universe_id, row["id"])
for batch in client.stream_data_batch(universe_id, collection["id"], include_content=True):
    print(batch)

For text or arbitrary JSON rows, supply raw_content=text.encode("utf-8"). For JSON, set metadata={"content_type": "application/json"}. Read responses represent rawContent as base64; decode with base64.b64decode(row["rawContent"]).

Updates replace only supplied fields. Passing metadata={} or labels={} clears that map; raw_content=b"" saves an intentionally empty row. Replacing llm_data replaces the whole structured example, so include every LLM field you want to keep. Labels and both metadata maps use string values; encode nested annotations as JSON strings. GetDataPoint returns inline content; list RPCs omit it to keep pages small.

RPCs: DataService.CreateDataPoint, GetDataPoint, UpdateDataPoint (with google.protobuf.FieldMask), and StreamDataBatch(include_content=true). Collections use CollectionService.AddDataToCollection, ListCollectionData, and RemoveDataFromCollection. Removing membership keeps the row itself. HTTP clients use POST/GET/PATCH /api/universes/{id}/data[/{row_id}] and the collection endpoints.

Create or reuse a collection while writing rows (SDK 0.1.6)

CreateDataPoint, UpdateDataPoint, and BatchCreateDataPoints accept either collection_name or collection_id. A name reuses the exact, case-sensitive name in the current workspace, creating it if absent. Names must be nonblank and at most 200 UTF-8 bytes. An ID must already exist in that workspace. Do not supply both. Omit both to retain the existing behavior.

row = client.create_data_point(universe_id, "example", raw_content=b"hello",
                               collection_name="Training examples")
client.update_data_point(universe_id, row["id"], raw_content=b"edited",
                         collection_name="Reviewed examples")
# Collection-only updates preserve content and other row fields.
client.update_data_point(universe_id, row["id"], collection_name="Training examples")
batch = client.batch_create_data_points(universe_id,
    ({"name": f"row-{i}", "raw_content": f"example {i}".encode()} for i in range(10000)),
    collection_name="Training examples")
collection_id = batch["collectionId"]

Each operation commits the row writes, optional collection creation, and membership in one PostgreSQL transaction. Assignment adds membership without removing existing memberships; repeat assignment does not duplicate it. Batch collection selection is at the request level and applies to every row. Responses include collection_id (collectionId in JSON and SDK results) when assigned. HTTP request fields are collectionName / collectionId; the RPC fields use snake_case. Direct RPC updates should use update_mask for explicit field replacement/clearing; a collection-only request needs no mask.

The batch supports up to 10,000 rows, subject to 1 MiB per row and 32 MiB serialized request size. Inline content is stored in PostgreSQL data_points; membership is stored in collection_data_points. Uploaded files continue to use object storage.

Evaluation results and version comparison (0.1.7)

Save immutable collection versions with create_collection_version, iterate source examples with iter_version_examples, upload externally computed model results with upload_evaluation, and compare with compare_collection_versions. See the complete evaluation guide for Colab examples and limits.