zen-fronts 1.0.4

Async Multi Objective Hyperparameter PBT (fast-cython, parameterless)

zen-fronts

Async multi-objective PBT-style population management with a fast Monte‑Carlo selection core.

The library is a facade (ZenFronts) around three concerns:

1. storing params / point lifecycle (including tombstones),
2. storing per-point criterion stats (mu/trend/sigma2, async updates),
3. periodic selection (refresh()) driven by a compiled MC ranking kernel.

---

Quick start (runnable demo)

pip install -e ".[dev,examples]"
python examples/demo_noisy_zdt1_snapshots.py --out out/demo --epochs 40

The demo writes PNG snapshots (2 panels per image):

• left: objective space (mu) + semi‑transparent ellipses from sigma2
• right: rank space (pareto_core stable ranks; the coordinates in which Pareto is computed)

---

How to run the main loop (operational semantics)

When is a point “ready”?

refresh() only considers points that are:

• active (not tombstone), and
• ready for every criterion.

This is a deliberate ready gate: if your objectives arrive asynchronously, you can keep calling update_crits(pid, {crit: value}, t=...) whenever a metric arrives, and call refresh() whenever you want to run selection. Only fully observed points participate.

What to do with new children?

perform_new() creates a child (via your mutator) and optionally removes a loser.

Important: the child is not ready until you evaluate it and call update_crits() for all criteria.

Recommended per-epoch pattern:

# 1) ingest metrics (sync evaluation OR async updates)
# 2) run selection
losers = zf.refresh(now=t)

for loser in losers:
    parent = zf.choose_parent(loser)
    child, _ = zf.perform_new(parent, looser=loser, remove_looser=True)

    # must evaluate child at least once, otherwise it stays invisible to refresh()
    vals = evaluate(zf.params(child))
    zf.update_crits(child, vals, t=t + 0.1)

Tombstones (remove_looser=True)

If you remove a loser, it becomes inactive:

• params(pid) raises KeyError
• info(pid) still works (history remains accessible)
• the last info(pid)["selection"] snapshot is frozen (not overwritten by future refresh() calls)

This makes it safe to keep IDs for logging/auditing.

---

Picking percentile and n_samples

Let N = number of active & ready points and k = ceil(percentile * N).

• percentile controls selection pressure.

  • Start with 0.2–0.3 for PBT-like loops.
  • For small N, keep k at least 2–3 to avoid jitter.

• n_samples controls Monte‑Carlo stability.

  • Typical: 128–256 (increase for noisy objectives).

If you enable per-point distribution summaries (median/q25/q75), quantiles can be computed:

• exactly (quantiles_mode_i=0),
• streaming P² (quantiles_mode_i=1), or
• auto by budget (quantiles_mode_i=2).

Contract guarantee: quantiles mode must not change winners/losers, mean/std, or quality score.

---

Selection stats contract (stable, versioned)

After refresh(), each active point gets a info(pid)["selection"] dict with schema metadata:

• schema_name = "zen_fronts.selection_stats"
• schema_version = "1.0.0"

Downstream code should validate/normalize the structure via:

from zen_fronts.selection.schema import validate_selection_stats

st = validate_selection_stats(zf.info(pid)["selection"])  # raises on incompatible schema

Backward-compatibility rule: newer minor/patch versions are accepted; newer major is rejected. Adding new fields is allowed.

---

Docs

• docs/how_to_run_cycle.md — full guidance for readiness, children handling, async criteria, and tuning.

---

Benchmarks

Two benchmark scripts live in examples/:

1) Core benchmark (MC ranking only)

Measures only the compiled Monte‑Carlo ranking kernel (no facade/store overhead):

python examples/bench_mc_rank.py --out out/bench_mc_rank.csv

2) End-to-end loop benchmark ("as in production")

Measures a realistic cycle:

• update criteria for all active points,
• refresh() (including build_matrices + MC core),
• replace losers (choose_parent + perform_new),
• update criteria for each new child.

python examples/bench_refresh.py --out out/bench_refresh.csv \
  --Ns 128,256 --Ss 64,128,256,512 --epochs 30 --reps 3

Key scaling intuition:

• time is ~linear in n_samples;
• time is ~quadratic in N (domination matrix is N×N).