PhaseSweep

PhaseSweep runs phase-chained hyperparameter sweeps from one ordinary YAML file. That file contains your trainer configuration, the metric, and the search plan. PhaseSweep materializes the configured trainer input for every trial, decides what to try next, persists each phase winner, and carries selected values forward as fixed inputs to later phases.

This is useful when a full joint sweep is too expensive or hard to interpret. For example, choose architecture depth, then tune learning rate, then regularization. The configuration guide explains the inheritance model and its tradeoffs.

How it works

The starter.yaml that phasesweep init writes searches depth before learning rate (abridged; the full file also sets storage, the working directory, and the metric extractor):

experiment: phasesweep_starter

# The trainer receives one complete generated YAML per trial.
trial_command: "python -m phasesweep.examples.fake_train --config {config_path}"

trainer_config:
  model:
    n_layers: 8
    dropout: 0.1
  optimizer:
    lr: 0.0003
    weight_decay: 0.05

metric:
  name: eval_loss
  goal: minimize

phases:
  - name: depth
    n_trials: 2
    sampler: { type: grid }
    search_space:
      model.n_layers: { type: categorical, choices: [6, 8] }

  - name: learning_rate
    inherits: [depth] # the winning model.n_layers is fixed here
    n_trials: 2
    sampler: { type: grid }
    search_space:
      optimizer.lr: { type: categorical, choices: [0.0001, 0.0003] }

The packaged fake trainer uses report_objective(...). Your own trainer can report through that helper, W&B, MLflow, ordinary JSON, or logs.

Install and try it

Requirements: Python 3.11+ and a Linux host for real runs. GPUs are optional.

pip install phasesweep
phasesweep --version

mkdir phasesweep-demo && cd phasesweep-demo
phasesweep init                     # writes the starter template, starter.yaml
phasesweep validate starter.yaml    # checks it without launching anything
phasesweep run starter.yaml         # runs four tiny toy-trainer trials

phasesweep init never overwrites an existing file. Use phasesweep init --list to choose a template, --template NAME to write it, and -o PATH to choose its destination. The templates run on the packaged toy trainer; replace the command, keys, and extractor with your own.

The run log, runs/phasesweep_starter/run.log beside the config, shows the phase chaining (abridged):

[depth/trial_0] python -m phasesweep.examples.fake_train --config .../trainer_config.yaml
phase=depth TRIAL trial=0 state=COMPLETE outcome=success value=0.3 duration=0.8s (1/2 finished)
[depth/trial_1] python -m phasesweep.examples.fake_train --config .../trainer_config.yaml
phase=depth TRIAL trial=1 state=COMPLETE outcome=success value=0.5 duration=0.7s (2/2 finished)
phase=depth WINNER trial=0 metric=0.3 params={'model.n_layers': 8}
[learning_rate/trial_0] python -m phasesweep.examples.fake_train --config .../trainer_config.yaml
phase=learning_rate TRIAL trial=0 state=COMPLETE outcome=success value=0.3 duration=0.8s (1/2 finished)
[learning_rate/trial_1] python -m phasesweep.examples.fake_train --config .../trainer_config.yaml
phase=learning_rate TRIAL trial=1 state=COMPLETE outcome=success value=0.413822 duration=0.8s (2/2 finished)
phase=learning_rate WINNER trial=0 metric=0.3 params={'optimizer.lr': 0.0003}

The depth winner's model.n_layers: 8 is injected into every learning_rate trial's complete YAML. Winners persist in the working directory, so you can inspect them any time (abridged):

phasesweep show-winners starter.yaml

phase: learning_rate
metric:
  eval_loss: 0.3
  goal: minimize
trial_number: 0
params:
  optimizer.lr: 0.0003
effective_overrides:
  model.n_layers: 8
  optimizer.lr: 0.0003
# ... completion state and evidence follow

Inspect recorded trials without launching work:

phasesweep status starter.yaml --trials

Run the same config again to resume after an interruption. See continuation and reruns for trial budgets and reproducibility, and inspection commands for status and result views.

Use your own trainer

Start from the template closest to your trainer. Choose its existing input interface: complete YAML, command-line flags, Hydra overrides, or nested JSON. phasesweep explain KEY prints the reference for one key; configuration errors name the relevant keys.

Set execution.cwd, env, and execution.inherit_env for the trainer's working directory and environment. validate and --dry-run print the launch contract so you can check it before training.

Choose an objective extractor for the trainer's output. For W&B, install the extra and follow the W&B setup:

pip install "phasesweep[wandb]"

For MLflow, install phasesweep[mlflow] and follow the MLflow guide. Use objective reporting when the checkpoint behind a value matters, and small-budget advice and repeats for noisy measurements.

Connect an agent

The optional MCP server connects an AI agent to experiments you have approved without exposing config or storage paths, trainer commands, environment values, or raw logs. Sampled winner values follow the catalog's visible_params policy. Follow the MCP setup to install the extra, review the catalog authority boundary, configure a client-owned connection, and verify the result; phasesweep mcp call runs one tool from a shell without a client.

Reference

• Configuration guide: Experiment YAML, search spaces, inheritance, noisy objectives and small budgets, repeats, gates, and objective extractors.
• Configuration reference: per-key types, defaults, valid values, interactions, and lifecycle warnings. Too long to read whole; phasesweep explain lists the top-level keys and phasesweep explain 'phases[].sampler' prints one key and everything below it.
• Minimal starting configuration: one local toy-trainer attempt; validate or run it from the checkout root.
• Runtime behavior: filesystem layout, locks, GPU leases, process supervision, fingerprints, and resume.
• MCP setup: installed-package agent onboarding and client-owned setup.
• MCP operator reference: catalog fields, tools, authorization, run state, and recovery.
• Toy experiment and MCP catalog: a checkout-local CLI example backed by the packaged fake trainer, plus an MCP catalog whose detached-run state and experiment outputs use absolute scratch paths under /tmp.
• Tiny Decoder Enwik8 example: real-trainer integration.
• Durability invariants: the ordering rules the ledger, publication, and recovery paths must hold, and the test that owns each one.
• Development: source checkout, contributor setup, and quality gates.

License

MIT. See LICENSE.