Dataerai Python SDK

Python client for the Dataerai transfer daemon.
Supports authenticated uploads, downloads, and metadata operations. The daemon is started automatically if it is not already running.

Requirements

Python ≥ 3.10
The dataerai binary must be installed and on PATH (or pass binary_path explicitly)
The user must be logged in via dataerai auth login before calling auth_status() / upload() / download()

Installation

pip install --pre "dataerai-sdk>=0.2.0b1,<0.3"
# or, from source:
pip install -e sdk/python/

Optional ML adapters are installed separately so the base SDK stays small:

pip install --pre "dataerai-sdk[ml]>=0.2.0b1,<0.3"          # PyTorch Zarr datasets
pip install --pre "dataerai-sdk[ml-keras]>=0.2.0b1,<0.3"    # Keras batches
pip install --pre "dataerai-sdk[ml-hls4ml]>=0.2.0b1,<0.3"   # HLS4ML conversion

Quick start

from dataerai import DataeraiClient

with DataeraiClient(binary_path="/usr/local/bin/dataerai") as client:
    # Check auth
    status = client.auth_status()
    print(f"Logged in as {status.user_email} ({status.user_id}), token expires {status.expires_at}")
    if status.user_id is None:
        raise RuntimeError("Upgrade the dataerai CLI to use user-owned uploads")

    # Upload a user-owned file. owner_id is a UUID, not an email address.
    result = client.upload(
        "/path/to/data.csv",
        title="My dataset",
        owner_type="user",
        owner_id=status.user_id,
        record_type="dataset",
        on_progress=lambda p: print(f"  {p.percent:.0f}%  {p.rate_mbps:.1f} MB/s"),
    )
    print(f"Uploaded  asset_id={result.asset_id}  content_id={result.content_id}")

    # Download it back
    dl = client.download(result.asset_id, dest_dir="/tmp/downloads")
    for f in dl.files:
        print(f"  {f.local_path}  ({f.size:,} bytes)")

    # Read / update metadata
    meta = client.get_metadata(result.asset_id)
    updated = client.set_metadata(
        result.asset_id,
        title="My dataset v2",
        record_type="analysis",
        tags=["csv", "demo"],
    )

    # Link a derived result to the source asset that produced it
    relationship = client.create_relationship(
        result.asset_id,
        "source-asset-id",
        relationship_type="derived_from",
        analysis_mode="non_destructive",
        qualifiers={"tool": "pycroscopy"},
    )
    related_id = relationship.related_asset["id"] if relationship.related_asset else "source-asset-id"
    print(f"Linked via {relationship.type} to {related_id}")

API reference

`DataeraiClient(*, socket_path, binary_path, auto_start, start_timeout_s, request_timeout_s)`

Parameter	Default	Description
`socket_path`	`$DATAERAI_SOCKET` or `/run/user/<uid>/dataerai-transfer.sock`	Unix socket path
`binary_path`	`None`	Path to the `dataerai` binary (required for `auto_start`)
`auto_start`	`True`	Spawn the daemon if the socket does not exist
`start_timeout_s`	`10.0`	Seconds to wait for the daemon socket to appear
`request_timeout_s`	`30.0`	Per-request timeout in seconds

Use as a context manager (with DataeraiClient(...) as client:) for automatic cleanup, or call client.connect() / client.close() manually.

Methods

Method	Returns	Description
`auth_status()`	`AuthStatus`	Logged-in user ID, email, and token expiry; `user_id` is `None` with older daemons
`create_project(name, *, description="")`	`Project`	Create a project (+ root collection, owner membership, default allocation); requires write scope
`upload(local_path, *, title, owner_type, owner_id, record_type=None, ...)`	`UploadResult`	Upload a file with an optional server-validated record type; blocks until complete
`download(asset_id, dest_dir, ...)`	`DownloadResult`	Download latest asset content; blocks until complete
`get_metadata(asset_id)`	`AssetMetadata`	Retrieve asset metadata
`set_metadata(asset_id, **fields)`	`AssetMetadata`	Update metadata fields, including `record_type`
`create_relationship(from_asset_id, to_asset_id, rel_type=None, *, relationship_type=None, ...)`	`Relationship`	Create a directed provenance link between two assets

For headless notebooks, pass record_type to upload() or set_metadata(). Both operations use the authenticated transfer daemon, so integrations do not need to read the CLI credential store or expose access tokens to Python.

ML adapters

The dataerai.ml package reads Dataerai-hosted Zarr image stores directly from the short-lived /zarr-access/ bundle. PyTorch users can keep using ZarrAssetDataset / ZarrOriginalsDataset; Keras users get matching batch datasets:

from dataerai.ml import access
from dataerai.ml.keras_dataset import ZarrOriginalsSequence

creds = access.load_credentials()
train = ZarrOriginalsSequence(
    creds,
    asset_id="dataset-asset-id",
    size=64,
    batch_size=32,
    shuffle=True,
    label_mode="categorical",
)

model.fit(train, epochs=5)

For HLS4ML conversion, keep inputs in Keras' channels-last layout:

from dataerai.ml.hls4ml import convert_keras_model

hls_model = convert_keras_model(
    model,
    output_dir="hls-project",
    project_name="dataerai_model",
    data=train,              # writes input/output .npy testbench arrays
    testbench_batches=2,
)

`create_relationship()` arguments

Authors a directed provenance edge from_asset_id → to_asset_id. You need write access to the source and read access to the target. rel_type is a free-form verb describing the source's role, e.g. "analysis_of" or "acquired_with". Notebook integrations can pass the same value with the keyword-only alias relationship_type.

Argument	Type	Description
`from_asset_id`	`str`	Source (dependent) asset — the edge starts here (required)
`to_asset_id`	`str`	Target (origin) asset — the edge points here (required)
`rel_type`	`str \| None`	Free-form relationship type, ≤255 chars (required unless `relationship_type` is provided)
`relationship_type`	`str \| None`	Keyword-only alias for `rel_type`
`analysis_mode`	`str \| None`	`non_destructive`, `altering`, `destructive`, `in_situ`, `ex_situ`, `invasive`, `non_invasive`
`qualifier_note`	`str \| None`	Free-text note on the relationship
`qualifier_time`	`str \| None`	ISO-8601 timestamp
`qualifiers`	`dict \| None`	JSON-serializable extra qualifiers

# Link a processed result back to the raw data it came from.
client.create_relationship(analysis.asset_id, raw.asset_id, "analysis_of",
                           analysis_mode="non_destructive")

`upload()` keyword arguments

Argument	Type	Description
`title`	`str`	Asset title (required)
`owner_type`	`str`	`"project"` or `"user"` (required)
`owner_id`	`str`	Owner entity ID (required)
`description`	`str \| None`	Free-text description
`alias`	`str \| None`	Short identifier
`tags`	`list[str] \| None`	Tag list
`metadata`	`dict \| None`	Arbitrary key-value metadata
`collection_id`	`str \| None`	Collection to add the asset to
`chunk_size_mb`	`int \| None`	Override default 64 MiB chunk size
`on_progress`	`Callable[[ProgressEvent], None] \| None`	Progress callback
`transfer_timeout_s`	`float`	Max seconds to wait for completion (default 3600)

Progress events

@dataclass
class ProgressEvent:
    transfer_id: str
    bytes_done: int
    bytes_total: int
    chunk_index: int
    chunk_count: int
    rate_bps: float
    file_index: int
    file_name: str

    @property
    def percent(self) -> float: ...   # 0–100

    @property
    def rate_mbps(self) -> float: ...

Error types

Exception	When raised
`DaemonError(code, message)`	Daemon returned a coded error (see `code` attribute)
`DaemonTimeoutError`	Request or transfer exceeded the configured timeout
`ConnectionError`	Daemon disconnected unexpectedly

dev bambu ml-io ml croissant ml-gpu-cu12 ml-gpu ml-keras ml-hls4ml ml-stream ml-stream-video ml-stream-train runner metaextract metaextract-converters metaextract-viz metaextract-checksums metaextract-utils metaextract-containers metaextract-microscopy metaextract-medical metaextract-bio metaextract-spectroscopy metaextract-all

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dataerai_sdk-0.2.0b12.tar.gz (568.7 kB view details)

Uploaded Jul 15, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

dataerai_sdk-0.2.0b12-py3-none-any.whl (521.3 kB view details)

Uploaded Jul 15, 2026 Python 3

File details

Details for the file dataerai_sdk-0.2.0b12.tar.gz.

File metadata

Download URL: dataerai_sdk-0.2.0b12.tar.gz
Upload date: Jul 15, 2026
Size: 568.7 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.2.0 CPython/3.11.2

File hashes

Hashes for dataerai_sdk-0.2.0b12.tar.gz
Algorithm	Hash digest
SHA256	`b6ecb7b4e864bbdde290c6de7457c6bb1602648b0adb6a3c00a2e166195e1e3f`
MD5	`147121d4039194ce56fd758eccf59b8e`
BLAKE2b-256	`6aac27677f10ece03acb055ced0fcb91515cf5e501a191025a06dd325b6d015e`

See more details on using hashes here.

File details

Details for the file dataerai_sdk-0.2.0b12-py3-none-any.whl.

File metadata

Download URL: dataerai_sdk-0.2.0b12-py3-none-any.whl
Upload date: Jul 15, 2026
Size: 521.3 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.2.0 CPython/3.11.2

File hashes

Hashes for dataerai_sdk-0.2.0b12-py3-none-any.whl
Algorithm	Hash digest
SHA256	`08024601deb4d1c1e870ec316ee356e98999e4951f4c664de1d706e68e596f3f`
MD5	`c832865c9c0931c60122f4e99c8cc7fd`
BLAKE2b-256	`374f838c6481060b90fa9717906706bcbda4ef2809e7cd144b4162e80cd9e626`