trade-study

Multi-objective trade-study orchestration: define factors, build parameter grids, run hierarchical study phases, and extract Pareto fronts — for any domain where you compare alternatives against competing objectives.

Statement of need

Comparing design alternatives against multiple objectives is a common task across scientific simulation, engineering trade-offs, and ML hyperparameter tuning. Researchers typically glue together separate tools for grid construction, execution, scoring, and Pareto analysis, writing ad-hoc scripts that are hard to reproduce or extend to multi-phase studies (screening → refinement → benchmark).

trade-study provides a single orchestration layer that composes these steps into a reproducible, protocol-driven workflow. Users supply a Simulator (generates data) and a Scorer (evaluates it); the framework handles grid construction, parallel execution, Pareto extraction, and phase chaining. All components are modular and optional — use only what you need.

This package targets researchers and practitioners who need:

• structured multi-phase experimental design (screen → refine → benchmark),
• multi-objective Pareto analysis across heterogeneous factors, and
• a reproducible Python API that separates domain logic from study orchestration.

Why trade-study?

Existing tools solve pieces of this problem — optuna for adaptive optimization, pymoo for multi-objective solvers, SALib for sensitivity analysis — but none provide the hierarchical phase orchestration that connects them into a single study.

Quick start

This wiring sketch assumes domain-specific MySimulator and MyScorer implementations like those shown under Protocols. See the worked guides in the documentation for complete, executable examples.

from trade_study import (
    Direction,
    Factor,
    FactorType,
    Observable,
    Phase,
    Study,
    build_grid,
    top_k_pareto_filter,
)

# 1. Define objectives
accuracy = Observable("accuracy", Direction.MAXIMIZE)
latency = Observable("latency_ms", Direction.MINIMIZE)
cost = Observable("cost_usd", Direction.MINIMIZE)

# 2. Define factors and build a design grid
factors = [
    Factor("learning_rate", FactorType.CONTINUOUS, bounds=(1e-4, 1e-1)),
    Factor("backend", FactorType.CATEGORICAL, levels=["A", "B", "C"]),
]
grid = build_grid(factors, method="lhs", n_samples=500)

# 3. Run a hierarchical study
study = Study(
    world=MySimulator(),  # implements Simulator protocol
    scorer=MyScorer(),  # implements Scorer protocol
    observables=[accuracy, latency, cost],
    phases=[
        Phase(
            "screening",
            grid=grid,
            filter_fn=top_k_pareto_filter(k=20),
        ),
        Phase("benchmark", grid="carry", filter_fn=None),
    ],
)
study.run(n_jobs=-1)

# 4. Inspect results
print(study.summary())
front = study.front("benchmark")  # non-dominated config indices
study.compare_phases()  # per-phase hypervolume and IGD+ between successive fronts

For a complete 15-minute demonstration with live code and saved figures, see the serosurvey design notebook and presentation guide. From a repository checkout, launch it with uv run --extra notebook jupyter lab examples/serosurvey_study.ipynb.

Protocols

Users implement two protocols to plug in their domain:

from trade_study import Scorer, Simulator


class MySimulator:
    """Implements the Simulator protocol."""

    def generate(self, config: dict) -> tuple:
        """Return (truth, observations) for a given config."""
        ...


class MyScorer:
    """Implements the Scorer protocol."""

    def score(self, truth, observations, config: dict) -> dict[str, float]:
        """Return {observable_name: value} for a single trial."""
        ...

Installation

pip install trade-study[all]

Or install only the extras you need:

pip install trade-study[design,pareto]

Core dependency: numpy only.

API overview

Design

from trade_study import Factor, FactorType, build_grid, screen

factors = [
    Factor("x", FactorType.CONTINUOUS, bounds=(0.0, 1.0)),
    Factor("method", FactorType.CATEGORICAL, levels=["a", "b", "c"]),
]

grid = build_grid(factors, method="sobol", n_samples=1024)
si = screen(run_fn, factors, method="morris", n_trajectories=3000)

Execution

from trade_study import run_grid, run_adaptive

# Grid mode: evaluate all configs (optional parallelism)
results = run_grid(world, scorer, grid, observables, n_jobs=-1)

# Adaptive mode: evolutionary multi-objective optimization (Optuna NSGA-II)
results = run_adaptive(world, scorer, factors, observables, n_trials=600)

Pareto analysis

from trade_study import extract_front, hypervolume, pareto_rank

front_idx = extract_front(results.scores, directions)
ranks = pareto_rank(results.scores, directions)
hv = hypervolume(results.scores[front_idx], ref_point, directions)

Stacking

from trade_study import stack_scores, stack_bayesian, stack_proportional, ensemble_predict

weights = stack_scores(score_matrix)  # simplex-constrained optimization
weights = stack_proportional(score_matrix)  # preserve weight across near ties
weights = stack_bayesian(idata_dict)  # arviz stacking (Yao et al. 2018)
combined = ensemble_predict(predictions, weights)

I/O

from trade_study import save_results, load_results

save_results(results, "study_results")
results = load_results("study_results")
frame = results.to_dataframe()  # optional pandas extra; includes trial metadata
frame.to_csv("study_results.csv", index=False)

Multi-fidelity search and surrogates

from trade_study import run_successive_halving, run_hyperband, fit_surrogate, fit_regime_surrogate

# Cheap-first budget-constrained screening
results = run_successive_halving(world, scorer, factors, observables, budget_param="maxiters")
results = run_hyperband(world, scorer, factors, observables, budget_param="maxiters")

# Interpolate scores from a completed results table (GP or RF)
surrogate = fit_surrogate(results, method="gp")

# Recommend a config per regime, interpolating over regime descriptors
regime_surrogate = fit_regime_surrogate(results, regime_cols=["n", "p"])

Surrogate fits report cross-validated cv_r2 and cv_rmse; check those diagnostics before trusting recommendations or post-hoc sensitivity results.

Constraint / feasibility_filter express infeasible regions of the design space; FactorConstraint couples factors during grid construction (e.g. keep x + y <= 1); screen() supports both Morris and Sobol sensitivity analysis. run_grid(..., n_reps=N) supports replicated trials, and sensitivity_from_table() estimates post-hoc sensitivity from an existing results table through a validated surrogate.

Release compatibility

Version 0.4.0 adds decision reporting (preference_sweep() with regret, practical equivalence and optional paired uncertainty), fixed-sample proportion intervals and replicate sizing, versioned result bundles with provenance, sequential replication (run_sequential()), evaluation caching, incremental grid checkpoints with bounded retries, grouped surrogate validation with support diagnostics, and the IVAC serosurvey notebook. The adaptive extra now requires Optuna 4.5 or later, and strict </> adaptive constraints treat exact equality as infeasible, matching result-table feasibility.

Version 0.3.0 adds replicated and paired studies, uncertainty diagnostics, persistent adaptive sessions, checkpointing, and DataFrame export. Existing phase-name-only checkpoints are refused; use a fresh checkpoint directory and set checkpoint_key to track model/data revisions. Confidence constraints require estimated standard errors rather than assuming unknown uncertainty is zero. stack_proportional() now uses exponential score utilities in both directions; set its temperature in score units to control concentration.

Related packages

Development

See CONTRIBUTING.md for setup, code standards, and the PR workflow.

uv sync --extra dev
just ci          # lint → mypy --strict → pytest with coverage
just format      # auto-format
just check       # auto-fix lint

Citation

If you use this package in your research, please cite:

@software{trade_study2026,
  author = {Macdonald, Joshua C.},
  title = {{trade-study}: Multi-Objective Trade-Study Orchestration},
  year = {2026},
  url = {https://github.com/jcm-sci/trade-study},
  version = {0.4.0},
  doi = {10.5281/zenodo.19599838},
}

See CITATION.cff for machine-readable metadata.

License

MIT