Dreamlake

A simple and flexible SDK for ML experiment tracking and data storage.

Features

• Three Usage Styles: Decorator, context manager, or direct instantiation
• Dual Operation Modes: Remote (API server) or local (filesystem)
• Auto-creation: Automatically creates namespace, workspace, and folder hierarchy
• Upsert Behavior: Updates existing episodes or creates new ones
• Simple API: Minimal configuration, maximum flexibility
• Time-Based Queries: MCAP-like API for querying track data by timestamp ranges
• Multi-Modal Sync: Timestamp inheritance for synchronizing pose, images, and sensor data

Installation

uv add dreamlake@0.4.2

pip install dreamlake==0.7.0

CLI (deprecated — use the standalone DreamLake CLI)

The Python CLI bundled in this package is deprecated, and this package no longer installs a dreamlake console script. Install the standalone DreamLake CLI instead — same commands, flags, and env vars (including artifact push and the workflow group), plus environment switching (dreamlake env use):

curl -fsSL https://dl.dreamlake.ai/install.sh | bash

The CLI code stays in this package for now. The internal artifact|workflow append-local writers are NOT deprecated — they remain the canonical DreamDB writers that dreamlake-server spawns, reachable via python -m dreamlake.cli (point WORKFLOWS_APPLY_BIN at a wrapper that invokes it).

Quick Start

Remote Mode (with API Server)

from dreamlake import Episode

with Episode(
    name="my-experiment",
    workspace="my-workspace",
    remote="https://cu3thurmv3.us-east-1.awsapprunner.com",
    api_key="your-jwt-token"
) as episode:
    print(f"Episode ID: {episode.id}")

Local Mode (Filesystem)

from dreamlake import Episode

with Episode(
    name="my-experiment",
    workspace="my-workspace",
    local_path=".dreamlake"
) as episode:
    pass  # Your code here

See examples/ for more complete examples.

Development Setup

Installing Dev Dependencies

To contribute to Dreamlake or run tests, install the development dependencies:

uv sync --extra dev

pip install -e ".[dev]"

This installs:

• pytest>=8.0.0 - Testing framework
• pytest-asyncio>=0.23.0 - Async test support
• sphinx>=7.2.0 - Documentation builder
• sphinx-rtd-theme>=2.0.0 - Read the Docs theme
• sphinx-autobuild>=2024.0.0 - Live preview for documentation
• myst-parser>=2.0.0 - Markdown support for Sphinx
• ruff>=0.3.0 - Linter and formatter
• mypy>=1.9.0 - Type checker

Running Tests

uv run pytest

pytest

Building Documentation

Documentation is built using Sphinx with Read the Docs theme.

uv run python -m sphinx -b html docs docs/_build/html

uv run sphinx-autobuild docs docs/_build/html

rm -rf docs/_build

The live preview command starts a local server and automatically rebuilds when files change.

Alternatively, you can use the Makefile from within the docs directory:

cd docs
make html          # Build HTML documentation
make clean         # Clean build files

For maintainers, to build and publish a new release: uv build && uv publish