expctl 0.2.0

Universal local-first experiment lifecycle manager inspired by Git

A universal, local-first experiment lifecycle manager inspired by Git.

English · Русская документация

Git tracks how code changes. expctl tracks how experiments happen.

expctl records the complete lifecycle of an experiment:

experiment
├── run → failed
├── change/fix → Git diff or commit
├── retry → completed
├── metrics and artifacts
└── conclusion

It is not tied to machine learning. It can register Python scripts, C++ benchmarks, shell commands, ROS launches, Docker containers, simulations, network tests, hardware measurements, and manual physical experiments.

Why expctl?

A Git commit tells you which code existed. It does not tell you:

• which command was executed;
• whether the process failed or was interrupted;
• what changed before the retry;
• which metrics and artifacts were produced;
• whether expected outputs were actually created;
• what conclusion was drawn from the result.

expctl stores that missing experiment history while using Git when it is available.

Core principles

• Universal: the core does not know what YOLO, PyTorch, ROS, or LiDAR are.
• Local-first: SQLite and human-readable JSON/Markdown records work offline.
• Git-aware, not Git-dependent: Git snapshots are captured when available.
• Immutable run history: retries and resumes create new linked runs.
• Validated success: exit code 0 is not sufficient when required outputs are configured.
• Portable: the first version needs no server or cloud account.

Installation

From PyPI

After the package is published:

pipx install expctl

or:

python3 -m pip install expctl

Directly from GitHub

pipx install git+https://github.com/CactysFedya/expctl.git

Development installation

git clone https://github.com/CactysFedya/expctl.git
cd expctl
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install -e ".[dev]"
pytest
expctl --version

Expected:

6 passed
expctl 0.2.0

Quick start

Initialize expctl inside any project:

expctl init --name my-project

Create an experiment:

expctl new \
  --name generic-baseline \
  --goal "Check the baseline implementation" \
  --hypothesis "The command produces a valid result" \
  --command "python3 examples/dummy_experiment.py --steps 5 --fail-at 3 --output demo_output" \
  --metrics "demo_output/metrics.json" \
  --required-file "demo_output/result.artifact" \
  --required-metric score \
  --tag demo

Run the current experiment:

expctl run

After fixing code or configuration:

expctl fix RUN-... \
  -m "Removed the intentional failure" \
  --commit

Retry with a corrected command:

expctl retry RUN-... -- \
  python3 examples/dummy_experiment.py \
    --steps 5 \
    --output demo_output

Validate, record an artifact, and conclude:

expctl validate RUN-...

expctl result RUN-... \
  --metric duration=2.4:s \
  --artifact demo_output/result.artifact \
  --copy

expctl conclude EXP-... \
  -m "The corrected implementation passed validation" \
  --complete \
  --outcome accepted

Manual physical experiment

Git and a child command are optional:

mkdir lidar-range-test && cd lidar-range-test
expctl init --name lidar-range-test

expctl new \
  --name outdoor-range \
  --goal "Measure maximum stable range" \
  --required-metric max_range \
  --tag hardware

expctl run --manual

Record the result later:

expctl result RUN-... \
  --mark-completed \
  --metric max_range=42.7:m \
  --metric packet_loss=0.8:percent \
  --artifact measurement.txt \
  --copy

Validation model

exit code != 0
→ failed

exit code = 0, but a required file or metric is missing
→ incomplete

exit code = 0 and all configured checks pass
→ completed

Commands in 0.2.0

Area	Commands
Project	init, config, doctor, status
Experiments	new, use, list, show, log, derive, archive
Runs	run, retry, resume, cancel, recover
Changes	change, fix, diff
Results	result, validate, conclude

Generic change types

fix, code, config, parameter, data,
environment, hardware, procedure, other

Examples:

expctl change RUN-... --type parameter \
  -m "Changed input size from 512 to 640"

expctl change RUN-... --type hardware \
  -m "Moved the test from Raspberry Pi to workstation"

expctl fix RUN-... \
  -m "Fixed an incorrect buffer boundary" \
  --commit

Generated data

project/
├── .expctl.toml
├── .expctl/
│   ├── expctl.db
│   └── current
└── experiments/
    ├── README.md
    └── 20260723-example-001/
        ├── README.md
        ├── experiment.json
        ├── changes/
        └── runs/
            └── RUN-.../
                ├── run.json
                ├── command.sh
                ├── environment.json
                ├── git.diff
                ├── stdout.log
                ├── stderr.log
                └── artifacts/

Demo

Run the complete failure/fix/retry workflow:

make demo

More details: docs/demo.md.

Security note

expctl executes commands supplied by the user and may copy artifacts. Do not run experiment definitions from untrusted repositories without reviewing the command and file paths first. See SECURITY.md.

Project status

0.2.0 is an alpha release intended for local experimentation and feedback. Database compatibility is not guaranteed until the first stable release.

Contributing

Bug reports, design proposals, documentation improvements, and pull requests are welcome. Read CONTRIBUTING.md before submitting changes.

License

MIT. See LICENSE.