Dreamlake

A simple and flexible SDK for ML experiment tracking and data storage.

Features

• Three Usage Styles: Decorator, context manager, or direct instantiation
• Dual Operation Modes: Remote (API server) or local (filesystem)
• Auto-creation: Automatically creates namespace, workspace, and folder hierarchy
• Upsert Behavior: Updates existing episodes or creates new ones
• Simple API: Minimal configuration, maximum flexibility
• Time-Based Queries: MCAP-like API for querying track data by timestamp ranges
• Multi-Modal Sync: Timestamp inheritance for synchronizing pose, images, and sensor data

Installation

uv add dreamlake@0.6.0

pip install dreamlake==0.6.0

CLI (deprecated)

The dreamlake console script bundled with this package is deprecated — the CLI has been reimplemented in TypeScript as @dreamlake/cli, which installs the same dreamlake bin with the same commands, flags, and env vars (including artifact push and the workflow group):

npm i -g @dreamlake/cli

The Python console script keeps working for now (it prints a pointer on stderr), but new features land in the TS CLI only. The internal artifact|workflow append-local writers are NOT deprecated — they remain the canonical DreamDB writers that dreamlake-server spawns.

Quick Start

Remote Mode (with API Server)

from dreamlake import Episode

with Episode(
    name="my-experiment",
    workspace="my-workspace",
    remote="https://cu3thurmv3.us-east-1.awsapprunner.com",
    api_key="your-jwt-token"
) as episode:
    print(f"Episode ID: {episode.id}")

Local Mode (Filesystem)

from dreamlake import Episode

with Episode(
    name="my-experiment",
    workspace="my-workspace",
    local_path=".dreamlake"
) as episode:
    pass  # Your code here

See examples/ for more complete examples.

Development Setup

Installing Dev Dependencies

To contribute to Dreamlake or run tests, install the development dependencies:

uv sync --extra dev

pip install -e ".[dev]"

This installs:

• pytest>=8.0.0 - Testing framework
• pytest-asyncio>=0.23.0 - Async test support
• sphinx>=7.2.0 - Documentation builder
• sphinx-rtd-theme>=2.0.0 - Read the Docs theme
• sphinx-autobuild>=2024.0.0 - Live preview for documentation
• myst-parser>=2.0.0 - Markdown support for Sphinx
• ruff>=0.3.0 - Linter and formatter
• mypy>=1.9.0 - Type checker

Running Tests

uv run pytest

pytest

Building Documentation

Documentation is built using Sphinx with Read the Docs theme.

uv run python -m sphinx -b html docs docs/_build/html

uv run sphinx-autobuild docs docs/_build/html

rm -rf docs/_build

The live preview command starts a local server and automatically rebuilds when files change.

Alternatively, you can use the Makefile from within the docs directory:

cd docs
make html          # Build HTML documentation
make clean         # Clean build files

For maintainers, to build and publish a new release: uv build && uv publish