multimodal-mllog 1.1.0

Experiment logbook pipeline: automatic per-run capture + on-demand reporting

multimodal-mllog

Experiment logbook pipeline for researchers training ML models with coding agents.

An external observer that automatically captures per-run facts (metrics, git state, transcript, optional MLflow info) as JSON records, and on demand renders a markdown logbook from them. A local dashboard provides a UI to browse runs and generate logbooks.

coding agent  -->  mllog capture (automatic via hook)  -->  local JSON store
researcher    -->  /logbook or dashboard (manual)       -->  markdown logbook

Architecture

   CODING AGENT  (Claude Code / Codex / ...)
   +----------------------------------------------------------------------+
   |  after each run: assemble facts -> write run record (automatic/hook) |
   |  on /logbook:    read records over window -> render doc              |
   +------+-------------------------------------------+------------------+
          | writes (automatic)                        | reads (manual)
          v                                           v
   CAPTURE (per run)                           REPORT (/logbook)
   * transcript, metrics, git, MLflow          * select records in window
   * one JSON per run                          * render markdown
          |                                    * advance checkpoint
          v
   LOCAL JSON STORE           <----->          DASHBOARD (React + Django)
   ./mllog/records/                            http://localhost:8000

Installation

pip install -e .                  # core (capture, query, render)
pip install -e ".[dashboard]"     # + dashboard (Django + React)
pip install -e ".[mlflow]"        # + optional MLflow integration

Quickstart

1. Open your ML project in a coding agent (Claude Code, Codex, etc.).
2. Run your experiment — train, evaluate, or analyze.
3. Run /mllog to capture the run, or let the Stop hook capture it automatically.
4. Run /logbook --from yesterday to generate a logbook from stored records.
5. Or start the dashboard: mllog dashboard and browse at http://localhost:8000.

CLI reference

# Capture a run
mllog capture --type {train|eval|analysis} --outcome {success|fail} \
              [--metrics-json <path>] [--transcript <path>] [--note <text>]

# Query records
mllog query --from <when> [--to <when>] [--json]
# <when>: ISO date (2026-06-25), 'yesterday', 'today'

# Render logbook
mllog render --from <when> [--to <when>] [--out <path>]

# Checkpoint
mllog checkpoint [--show | --advance]

# Dashboard
mllog dashboard [--host 127.0.0.1] [--port 8000]

Dashboard

A local React + Django dashboard to browse run records and generate logbooks.

pip install -e ".[dashboard]"
mllog dashboard
# Open http://localhost:8000

Features:

• Browse run records with date, type, and outcome filters
• View run details: metrics, git info, MLflow info, full transcript
• List and view generated logbooks
• Generate logbooks from the UI with a date range picker

Frontend development:

cd dashboard/frontend
npm install
npm run dev        # Vite HMR at http://localhost:5173

Schema

RunRecord (pydantic v2) — one JSON per run:

{
  "run_id": "uuid",
  "created_at": "ISO-8601 UTC",
  "started_at": "ISO-8601 UTC | null",
  "type": "train|eval|analysis",
  "outcome": "success|fail",
  "metrics": { "name": value },
  "metrics_source": "mlflow|session|none",
  "git": { "commit": "sha", "dirty": false, "changed_files": [] },
  "mlflow": { "run_id": "...", "params": {} },
  "transcript": [{ "role": "user|agent", "content": "..." }],
  "notes": "string|null"
}

Storage

• Records: ./mllog/records/<YYYY-MM-DD>/<run_id>.json
• Checkpoint: ./mllog/checkpoint.json
• Logbooks: ./mllog/logbooks/<from>_<to>.md
• Override root with MLLOG_DIR env var.
• MLflow is optional enrichment, not required.

Development

git clone https://github.com/jean-johnson-zwix/multimodal_mllogger
cd multimodal_mllogger
make install         # create venv + install deps
make test            # run pytest
make dashboard-dev   # start dashboard in dev mode
make dashboard-build # build React frontend

Roadmap

• v1.0: MLflow-centric pipeline with /mllog command.
• v1.1 (current): Local JSON store, automatic capture via SessionStart + Stop hooks, /logbook reporting, React + Django dashboard.
• v1.2: NoSQL/S3 storage, synthesis layer.

License

MIT