Dreamlake

A simple and flexible SDK for ML experiment tracking and data storage.

Features

Three Usage Styles: Decorator, context manager, or direct instantiation
Dual Operation Modes: Remote (API server) or local (filesystem)
Auto-creation: Automatically creates namespace, workspace, and folder hierarchy
Upsert Behavior: Updates existing episodes or creates new ones
Simple API: Minimal configuration, maximum flexibility
Time-Based Queries: MCAP-like API for querying track data by timestamp ranges
Multi-Modal Sync: Timestamp inheritance for synchronizing pose, images, and sensor data

Installation

Using uv (recommended)	Using pip
uv add dreamlake@0.15.0	pip install dreamlake==0.8.0

CLI (deprecated — use the standalone DreamLake CLI)

The Python CLI bundled in this package is deprecated, and this package no longer installs a dreamlake console script. Install the standalone DreamLake CLI instead — same commands, flags, and env vars, plus environment switching (dreamlake env use):

curl -fsSL https://dl.dreamlake.ai/install.sh | bash

upload, download, list, create, delete, update and vectorize have been removed from this package; they live in the standalone CLI. What is still here, reachable through python -m dreamlake.cli:

Command	Status
`artifact append-local`, `workflow append-local`	not deprecated — see below
`artifact push\|list\|delete\|restore`	deprecated; the standalone CLI has these
`workflow push\|list`	deprecated; the standalone CLI has these
`source create\|collection create\|push`	kept — no standalone-CLI equivalent yet
`video upload\|download\|list`	deprecated; the standalone CLI has these
`login`, `logout`, `profile`	kept — `login` is the only writer of the token store `artifact push` / `workflow push` read

The `append-local` writers are not deprecated

artifact append-local and workflow append-local are the canonical DreamDB writers, and dreamlake-server spawns them as a subprocess — routes/workflows.ts for workflow apply, routes/artifacts.ts for artifact apply. Their stdout is a machine contract: exactly one JSON line, which the server parses as the last non-empty line of stdout. Errors go to stdout too, as JSON. Do not reformat either.

workflow append-local  →  {"version": N, "meta": {description, stageCount, nodeCount, edgeCount}}
artifact append-local  →  {"version": N}            # no "meta" key — the two are not symmetric
error (either)         →  {"error": "...", "message": "..."}   # exit 1

The wrapper bin

The server spawns ${WORKFLOWS_APPLY_BIN:-dreamlake} workflow append-local … and ${ARTIFACTS_APPLY_BIN:-…} artifact append-local …. The dreamlake on PATH is the standalone TS CLI, which has no DreamDB writer, and pip install dreamlake deliberately installs no bin of its own. So point both env vars at the wrapper shipped in this repo:

export WORKFLOWS_APPLY_BIN=/path/to/dreamlake-py/bin/dreamlake-append-local
export ARTIFACTS_APPLY_BIN=/path/to/dreamlake-py/bin/dreamlake-append-local

It is a one-line exec python3 -m dreamlake.cli "$@" and must stay one — anything the wrapper prints would corrupt the JSON contract above. It is not registered in [project.scripts], on purpose: the dreamlake name on PATH belongs to the standalone CLI.

Quick Start

Remote Mode (with API Server)

from dreamlake import Episode

with Episode(
    name="my-experiment",
    workspace="my-workspace",
    remote="https://cu3thurmv3.us-east-1.awsapprunner.com",
    api_key="your-jwt-token"
) as episode:
    print(f"Episode ID: {episode.id}")

Local Mode (Filesystem)

from dreamlake import Episode

with Episode(
    name="my-experiment",
    workspace="my-workspace",
    local_path=".dreamlake"
) as episode:
    pass  # Your code here

See examples/ for more complete examples.

Development Setup

Installing Dev Dependencies

To contribute to Dreamlake or run tests, install the development dependencies:

Using uv (recommended)	Using pip
uv sync --extra dev	pip install -e ".[dev]"

This installs:

pytest>=8.0.0 - Testing framework
pytest-asyncio>=0.23.0 - Async test support
sphinx>=7.2.0 - Documentation builder
sphinx-rtd-theme>=2.0.0 - Read the Docs theme
sphinx-autobuild>=2024.0.0 - Live preview for documentation
myst-parser>=2.0.0 - Markdown support for Sphinx
ruff>=0.3.0 - Linter and formatter
mypy>=1.9.0 - Type checker

Running Tests

Using uv	Using pytest directly
uv run pytest	pytest

Building Documentation

Documentation is built using Sphinx with Read the Docs theme.

Build docs	Live preview	Clean build
uv run python -m sphinx -b html docs docs/_build/html	uv run sphinx-autobuild docs docs/_build/html	rm -rf docs/_build

The live preview command starts a local server and automatically rebuilds when files change.

Alternatively, you can use the Makefile from within the docs directory:

cd docs
make html          # Build HTML documentation
make clean         # Clean build files

For maintainers, to build and publish a new release: uv build && uv publish

Release files for dreamlake 0.17.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dreamlake 0.17.0
File	Size	Uploaded
dreamlake-0.17.0.tar.gz	214.4 kB	Sep 15, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for dreamlake 0.17.0
File	Interpreter	ABI	Platform	Reset
dreamlake-0.17.0-py3-none-any.whl 256.9 kB Sep 15, 2026	Python 3	none	any	Details

Total release size: 471.3 kB

Release files / dreamlake-0.17.0.tar.gz

Download URL	dreamlake-0.17.0.tar.gz
Size	214.4 kB
Tags	Source
SHA-256 checksum How to use checksums	`30bb2f1a49c2c8416a244ad25555a4e1f033a76ad2189d6972cb4cb9101e2b3f`
BLAKE2b-256 checksum How to use checksums	`314b2d3918fac286eaf7bded950b0c03f68bcc785cd472a04300f39a3546aad5`
Upload date	Sep 15, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.2 {"installer":{"name":"uv","version":"0.12.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}`

Release files / dreamlake-0.17.0-py3-none-any.whl

Download URL	dreamlake-0.17.0-py3-none-any.whl
Size	256.9 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`2b0d3daf2d573e178a23042a82ca59a2d0e81d7e5d56cc74fd8327333c1d1989`
BLAKE2b-256 checksum How to use checksums	`ba75e127b43c6ec613c79b6a66eeb7a3cfec15ed461b7eecf0a70c17e906adf4`
Upload date	Sep 15, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.2 {"installer":{"name":"uv","version":"0.12.2","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}`