arrowspace_tuner

Hyperparameter discovery for ArrowSpace — automatically finds the best eps, k, and tau for your corpus using a query-free spectral objective.

Why

ArrowSpace's retrieval quality depends on three parameters:

Setting these by hand is tedious and corpus-dependent. arrowspace_tuner uses Optuna and a label-free spectral MRR proxy to find them automatically in minutes.

Install

# Core (no pandas/plotly)
pip install arrowspace-tuner

# With HTML/CSV reporting
pip install arrowspace-tuner[report]

Quickstart

Executable versions of these snippets live in examples/quickstart.py and examples/power_user.py, and are run on every CI build.

import numpy as np
import arrowspace_tuner
from arrowspace import ArrowSpaceBuilder   # builder comes from `arrowspace`

embeddings = np.load("corpus.npy")   # shape (N, D) float64

# One-liner: auto-discover eps, k, tau — runs in ~15 min on 50k corpus
graph_params = arrowspace_tuner.tune(embeddings)

# The caller owns the build step
aspace, gl = ArrowSpaceBuilder().build(graph_params, embeddings)

# Search as normal — tau is a query-time parameter
results = aspace.search(query_embedding, gl, tau=0.8)

Build-time vs. search-time parameters

graph_params:
  Build-time parameters only.
  Expected native ArrowSpace keys:
  eps, k, topk, p, sigma.

best_tau:
  Search-time parameter.
  It is intentionally excluded from graph_params.

Every public result dictionary — from tune(), EpsTuner.fit(), EpsTuner.graph_params, and load_graph_params() — uses the bindings-native topk key and can be passed verbatim to ArrowSpaceBuilder().build(). best_tau is a separate search-time result: use it at query time as aspace.search(q, gl, tau=tuner.best_tau).

Power-user API

Executable version: examples/power_user.py.

from arrowspace import ArrowSpaceBuilder
from arrowspace_tuner import EpsTuner

tuner = EpsTuner(
    n_trials  = 15,
    seed      = 42,
    sample_n  = 50_000,
    eps_low   = 0.8,
    eps_high  = 10,
    k_low     = 15,
    k_high    = 40,
    n_probe   = 50,
    storage   = "sqlite:///tune.db",   # resume interrupted runs
)

graph_params = tuner.fit(embeddings)
best_tau = tuner.best_tau           # query-time only — not in graph_params

# The caller owns the build step
aspace, gl = ArrowSpaceBuilder().build(graph_params, embeddings)

print(graph_params)          # {"eps": 1.615, "k": 38, "topk": 19, "p": ..., "sigma": ...}
print(tuner.best_tau)        # 0.114  — query-time only, not in graph_params
print(tuner.best_score)      # 2.138
print(tuner.best_fiedler)    # 0.718  — graph connectivity health
print(tuner.best_mrr_proxy)  # 2.896  — retrieval coherence proxy

# Access graph params without file I/O
print(tuner.graph_params)    # same dict as best_params, raises RuntimeError before .fit()

# Save CSV + HTML plots (requires [report] extra)
tuner.save_report(out_dir="results")

The final build after the study always uses the full corpus.

Objective

The objective is a weighted composite of three spectral signals — no ground-truth labels required:

score = 0.70 * mrr_top0_spectral   # retrieval coherence
      + 0.20 * log1p(fiedler)      # graph connectivity health
      + 0.10 * log1p(var_lambda)   # spectral richness

Parallel runs

Optuna + SQLite lets you run multiple workers simultaneously:

# Terminal 1
python -m arrowspace_tuner --storage sqlite:///tune.db --trials 15

# Terminal 2 (simultaneously)
python -m arrowspace_tuner --storage sqlite:///tune.db --trials 15

Requirements

• Python ≥ 3.12
• arrowspace >= 0.26.0, < 0.29 — tested with 0.26.0, 0.27.3, and 0.28.1
• optuna >= 4.8.0
• scipy >= 1.17.1
• numpy >= 2.4.4

License

Apache-2.0 — see LICENSE.