experimaestro 2.6.0

Experimaestro is a computer science experiment manager

---

Experimaestro is a Python framework designed for researchers and engineers who need to manage complex, large-scale experimental workflows without losing track of reproducibility.

Unlike traditional schedulers, Experimaestro focuses on the experimental logic: how configurations relate to each other and how results are organized.

Why Experimaestro?

• 🧩 Configuration-as-Code: Define your experiments using strongly-typed Python objects. Forget about fragile JSON/YAML files; benefit from IDE autocompletion, type checking, and recursive parameter management.
• 🛡️ Deduplication & Reproducibility: Every task is assigned a unique identifier based on its parameters. If you try to run the same experiment twice, Experimaestro knows—ensuring you never waste compute time on results you already have.
• 📁 Organized by Design: Results are automatically cached in a predictable directory structure derived from task identifiers. No more "results_v2_final_fixed.pt"—your file system stays as clean as your code.
• 🏗️ Built-in Scalability: Seamlessly transition from local testing to high-performance clusters. Use Connectors (Local, SSH) and Launchers (Direct, Slurm) to run the same experimental code across different environments.
• 📺 Real-time Monitoring: Track running and completed experiments as they progress, from a textual (terminal) UI or a web UI.

Documentation

The full documentation is at experimaestro-python.readthedocs.io:

• Tutorial — set up your first workspace and run a basic experiment (training a CNN on MNIST).
• Configurations & Tasks — define parameters, dependencies and execution logic.
• Launchers & Connectors — control where and how your code runs.
• How it differs from Slurm, OAR, Comet, Sacred and other experiment managers.

Screenshots

Textual interface (new in v2)

Experiments overview: monitor (local or SSH) running and completed experiments

Jobs view: track job status, progress, and dependencies

Job details: inspect individual job parameters and output

Logs view: real-time log streaming for running tasks

Services view: monitor background services and their status

Web interface

Tasks view: filter, sort and monitor jobs by status, tags, duration and CO₂, with live progress and per-job logs/actions

Install

With pip

You can then install the package using pip install experimaestro

Develop

Checkout the git directory, then

pip install -e .

Coding assistant skill

Experimaestro ships an agent skill that teaches LLM coding assistants (Claude Code, Cursor, …) the framework's conventions and best practices. Install it with:

# Default: ~/.agents/skills/ (cross-client open standard)
experimaestro install-skill

# Install for a specific tool
experimaestro install-skill claude     # ~/.claude/skills/
experimaestro install-skill cursor     # ~/.cursor/skills/

# Install to several targets at once
experimaestro install-skill agents claude

# List available targets and what is already installed
experimaestro install-skill --list

Example

This very simple example shows how to submit two tasks that concatenate two strings. Under the curtain,

• A directory is created for each task (in workdir/jobs/helloworld.add/HASHID) based on a unique ID computed from the parameters
• Two processes for Say are launched (there are no dependencies, so they will be run in parallel)
• A tag y is created for the main task

# --- Task and types definitions

import logging
logging.basicConfig(level=logging.DEBUG)
from pathlib import Path
from experimaestro import Task, Param, experiment, progress
import click
import time
import os
from typing import List

# --- Just to be able to monitor the tasks

def slowdown(sleeptime: int, N: int):
    logging.info("Sleeping %ds after each step", sleeptime)
    for i in range(N):
        time.sleep(sleeptime)
        progress((i+1)/N)


# --- Define the tasks

class Say(Task):
    word: Param[str]
    sleeptime: Param[float]

    def execute(self):
        slowdown(self.sleeptime, len(self.word))
        print(self.word.upper(),)

class Concat(Task):
    strings: Param[List[Say]]
    sleeptime: Param[float]

    def execute(self):
        says = []
        slowdown(self.sleeptime, len(self.strings))
        for string in self.strings:
            with open(string.__xpm_stdout__) as fp:
                says.append(fp.read().strip())
        print(" ".join(says))


# --- Defines the experiment

@click.option("--port", type=int, default=12345, help="Port for monitoring")
@click.option("--sleeptime", type=float, default=2, help="Sleep time")
@click.argument("workdir", type=Path)
@click.command()
def cli(port, workdir, sleeptime):
    """Runs an experiment"""
    # Sets the working directory and the name of the xp
    with experiment(workdir, "helloworld", port=port) as xp:
        # Submit the tasks
        hello = Say.C(word="hello", sleeptime=sleeptime).submit()
        world = Say.C(word="world", sleeptime=sleeptime).submit()

        # Concat will depend on the two first tasks
        Concat.C(strings=[hello, world], sleeptime=sleeptime).tag("y", 1).submit()


if __name__ == "__main__":
    cli()

which can be launched with python test.py /tmp/helloworld-workdir

Ecosystem

A number of libraries and tools are built around experimaestro:

Datasets — datamaestro, a companion dataset manager, with plugins datamaestro_text, datamaestro_image, datamaestro_ml and datamaestro_ir.

Domain libraries

• xpm-torch — building blocks for PyTorch-based experiments.
• experimaestro-ir (xpmir) — information-retrieval tasks and configurations (built on xpm-torch). Specific experiments are their own projects in the xpmir organisation, e.g. splade, cosplade, cross-encoders and mice.

Tools & services — xpm-mlboard, lightweight services to monitor ML learning curves (TensorBoard, …).

Starting points — experiment-template (minimal skeleton) and experimaestro-demo (fuller MNIST example, also the tutorial).

See the Experimaestro projects guide for how to structure your own project.