trade-study

Multi-objective trade-study orchestration: define factors, build parameter grids, run hierarchical study phases, and extract Pareto fronts — for any domain where you compare alternatives against competing objectives.

Statement of need

Comparing design alternatives against multiple objectives is a common task across scientific simulation, engineering trade-offs, and ML hyperparameter tuning. Researchers typically glue together separate tools for grid construction, execution, scoring, and Pareto analysis, writing ad-hoc scripts that are hard to reproduce or extend to multi-phase studies (screening → refinement → benchmark).

trade-study provides a single orchestration layer that composes these steps into a reproducible, protocol-driven workflow. Users supply a Simulator (generates data) and a Scorer (evaluates it); the framework handles grid construction, parallel execution, Pareto extraction, and phase chaining. All components are modular and optional — use only what you need.

This package targets researchers and practitioners who need:

• structured multi-phase experimental design (screen → refine → benchmark),
• multi-objective Pareto analysis across heterogeneous factors, and
• a reproducible Python API that separates domain logic from study orchestration.

Why trade-study?

Existing tools solve pieces of this problem — optuna for adaptive optimization, pymoo for multi-objective solvers, SALib for sensitivity analysis — but none provide the hierarchical phase orchestration that connects them into a single study.

Quick start

This wiring sketch assumes domain-specific MySimulator and MyScorer implementations like those shown under Protocols. See the worked guides in the documentation for complete, executable examples.

from trade_study import (
    Direction,
    Factor,
    FactorType,
    Observable,
    Phase,
    Study,
    build_grid,
    top_k_pareto_filter,
)

# 1. Define objectives
accuracy = Observable("accuracy", Direction.MAXIMIZE)
latency = Observable("latency_ms", Direction.MINIMIZE)
cost = Observable("cost_usd", Direction.MINIMIZE)

# 2. Define factors and build a design grid
factors = [
    Factor("learning_rate", FactorType.CONTINUOUS, bounds=(1e-4, 1e-1)),
    Factor("backend", FactorType.CATEGORICAL, levels=["A", "B", "C"]),
]
grid = build_grid(factors, method="lhs", n_samples=500)

# 3. Run a hierarchical study
study = Study(
    world=MySimulator(),  # implements Simulator protocol
    scorer=MyScorer(),  # implements Scorer protocol
    observables=[accuracy, latency, cost],
    phases=[
        Phase(
            "screening",
            grid=grid,
            filter_fn=top_k_pareto_filter(k=20),
        ),
        Phase("benchmark", grid="carry", filter_fn=None),
    ],
)
study.run(n_jobs=-1)

# 4. Inspect results
print(study.summary())
front = study.front("benchmark")  # non-dominated config indices
study.compare_phases()  # per-phase hypervolume and IGD+ between successive fronts

For a complete 15-minute demonstration with live code and saved figures, see the serosurvey design notebook and presentation guide. From a repository checkout, launch it with uv run --extra notebook jupyter lab examples/serosurvey_study.ipynb.

Protocols

Users implement two protocols to plug in their domain:

from trade_study import Scorer, Simulator


class MySimulator:
    """Implements the Simulator protocol."""

    def generate(self, config: dict) -> tuple:
        """Return (truth, observations) for a given config."""
        ...


class MyScorer:
    """Implements the Scorer protocol."""

    def score(self, truth, observations, config: dict) -> dict[str, float]:
        """Return {observable_name: value} for a single trial."""
        ...

Installation

pip install trade-study[all]

Or install only the extras you need:

pip install trade-study[design,pareto]

Core dependency: numpy only.

API overview

Design

from trade_study import Factor, FactorType, build_grid, screen

factors = [
    Factor("x", FactorType.CONTINUOUS, bounds=(0.0, 1.0)),
    Factor("method", FactorType.CATEGORICAL, levels=["a", "b", "c"]),
]

grid = build_grid(factors, method="sobol", n_samples=1024)
si = screen(run_fn, factors, method="morris", n_trajectories=3000)

Execution

from trade_study import run_grid, run_adaptive

# Grid mode: evaluate all configs (optional parallelism)
results = run_grid(world, scorer, grid, observables, n_jobs=-1)

# Adaptive mode: evolutionary multi-objective optimization (Optuna NSGA-II)
results = run_adaptive(world, scorer, factors, observables, n_trials=600)

Pareto analysis

from trade_study import extract_front, hypervolume, pareto_rank

front_idx = extract_front(results.scores, directions)
ranks = pareto_rank(results.scores, directions)
hv = hypervolume(results.scores[front_idx], ref_point, directions)

Stacking

from trade_study import stack_scores, stack_bayesian, stack_proportional, ensemble_predict

weights = stack_scores(score_matrix)  # simplex-constrained optimization
weights = stack_proportional(score_matrix)  # preserve weight across near ties
weights = stack_bayesian(idata_dict)  # arviz stacking (Yao et al. 2018)
combined = ensemble_predict(predictions, weights)

I/O

from trade_study import save_results, load_results

save_results(results, "study_results")
results = load_results("study_results")
frame = results.to_dataframe()  # optional pandas extra; includes trial metadata
frame.to_csv("study_results.csv", index=False)

Multi-fidelity search and surrogates

from trade_study import run_successive_halving, run_hyperband, fit_surrogate, fit_regime_surrogate

# Cheap-first budget-constrained screening
results = run_successive_halving(world, scorer, factors, observables, budget_param="maxiters")
results = run_hyperband(world, scorer, factors, observables, budget_param="maxiters")

# Interpolate scores from a completed results table (GP or RF)
surrogate = fit_surrogate(results, method="gp")

# Recommend a config per regime, interpolating over regime descriptors
regime_surrogate = fit_regime_surrogate(results, regime_cols=["n", "p"])

Surrogate fits report cross-validated cv_r2 and cv_rmse; check those diagnostics before trusting recommendations or post-hoc sensitivity results.

Constraint / feasibility_filter express infeasible regions of the design space; FactorConstraint couples factors during grid construction (e.g. keep x + y <= 1); screen() supports both Morris and Sobol sensitivity analysis. run_grid(..., n_reps=N) supports replicated trials, and sensitivity_from_table() estimates post-hoc sensitivity from an existing results table through a validated surrogate.

Release compatibility

Version 0.5.0 adds a prespecified fixed replication policy and versioned replication-contract metadata. Adaptive allocation remains the default. Pareto operations now identify the optional pareto extra when it is missing.

Version 0.4.0 adds decision reporting (preference_sweep() with regret, practical equivalence and optional paired uncertainty), fixed-sample proportion intervals and replicate sizing, versioned result bundles with provenance, sequential replication (run_sequential(), with explicit fixed/adaptive policy and recorded counts/uncertainty assumptions), evaluation caching, incremental grid checkpoints with bounded retries, grouped surrogate validation with support diagnostics, and the IVAC serosurvey notebook. The adaptive extra now requires Optuna 4.5 or later, and strict </> adaptive constraints treat exact equality as infeasible, matching result-table feasibility.

Version 0.3.0 adds replicated and paired studies, uncertainty diagnostics, persistent adaptive sessions, checkpointing, and DataFrame export. Existing phase-name-only checkpoints are refused; use a fresh checkpoint directory and set checkpoint_key to track model/data revisions. Confidence constraints require estimated standard errors rather than assuming unknown uncertainty is zero. stack_proportional() now uses exponential score utilities in both directions; set its temperature in score units to control concentration.

Related packages

Development

See CONTRIBUTING.md for setup, code standards, and the PR workflow.

uv sync --extra dev
just ci          # lint → mypy --strict → pytest with coverage
just format      # auto-format
just check       # auto-fix lint

Citation

If you use this package in your research, please cite:

@software{trade_study2026,
  author = {Macdonald, Joshua C.},
  title = {{trade-study}: Multi-Objective Trade-Study Orchestration},
  year = {2026},
  url = {https://github.com/jcm-sci/trade-study},
  version = {0.5.0},
  doi = {10.5281/zenodo.19599838},
}

See CITATION.cff for machine-readable metadata.

License

MIT