Skip to main content

Adaptive Greedy Search (AGS)

Adaptive Greedy Search (AGS) v3.6.0 is a surrogate-assisted, multi-start adaptive local-search optimizer for finite discrete hyperparameter spaces.

It combines typed search-space geometry, a shared RF/GP/KDE surrogate, adaptive trust-region climbers, fold-level multi-fidelity racing, and diversity-aware batch/restart logic.

from ags import AdaptiveGreedySearch

What's new in 3.6

v3.4 — Adaptive trust-region climbers

Each climber now keeps separate current and historical-best state:

  • current_state, current_score
  • personal_best_state, personal_best_score
  • trust_radius

Confirmed improvement expands the local region; failed or pruned proposals contract it.

search = AdaptiveGreedySearch(
    estimator,
    param_grid,
    initial_trust_radius=1.0,
    min_trust_radius=1.0,
    trust_expand=2.0,
    trust_shrink=0.5,
    max_neighbor_radius=4,
)

v3.5 — Multi-fidelity confidence racing

CV folds are treated as fidelity. In addition to percentile, optimistic, and none, AGS now supports:

pruning_strategy="racing"
racing_confidence=0.95

After enough folds, a statistically weak candidate can be stopped when its confidence upper bound falls below the confirmed incumbent. Racing is a statistical resource-allocation rule, not a deterministic guarantee. Partial observations still cannot become the winner or train the surrogate as full observations.

v3.6 — Diversity-aware batch search and restart

Batch candidates are coordinated before evaluation. AGS adds a typed-distance diversity bonus to acquisition scores so parallel climbers are less likely to collapse into the same basin.

Retired climbers are respawned using both predicted quality and distance from active search regions.

search = AdaptiveGreedySearch(
    estimator,
    param_grid,
    diversity_weight=0.25,
    restart_diversity_weight=0.5,
)

Installation

pip install adaptive-greedy-search

Quickstart

from ags import AdaptiveGreedySearch
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification

X, y = make_classification(n_samples=500, n_features=20, random_state=0)

param_grid = {
    "n_estimators": [50, 100, 150, 200],
    "max_depth": [3, 5, 7, 9, None],
    "min_samples_split": [2, 4, 6, 8],
}

search = AdaptiveGreedySearch(
    RandomForestClassifier(random_state=0),
    param_grid,
    cv=5,
    scoring="accuracy",
    max_evaluations=30,
    n_climbers=6,
    n_jobs=-1,
    surrogate_type="rf",
    acquisition="auto",
    pruning_strategy="racing",
    random_state=42,
)

search.fit(X, y)

print(search.best_params)
print(search.best_score)
print("evaluations:", search.n_evaluations)
print("folds saved:", search.folds_saved)
print("mean fidelity:", search.mean_fidelity)
print("trust expansions:", search.trust_expansions)
print("trust contractions:", search.trust_contractions)

Typed search spaces

Plain lists still work. For explicit geometry:

from ags import NumericParameter, OrdinalParameter, CategoricalParameter

param_grid = {
    "C": NumericParameter([0.01, 0.1, 1, 10], scale="log"),
    "depth": OrdinalParameter([2, 4, 6, 8]),
    "kernel": CategoricalParameter(["linear", "rbf", "poly"]),
}

Numeric distance is normalized in value space, ordinal distance uses rank, and nominal categories use Hamming distance plus one-hot surrogate features.

Surrogates and acquisitions

Surrogate Meaning
rf Default. Random Forest mean + between-tree dispersion proxy
gp Gaussian Process predictive mean/std
kde Good/bad KDE density-ratio ranking
tpe Deprecated compatibility alias for kde; not a full TPE

Acquisition options are auto, ucb, ei / expected_improvement, and density_ratio where compatible.

The surrogate is trained only on complete CV evaluations after the warm-up target is reached.

Pruning / racing

Strategy Meaning
percentile Heuristic comparison with historical partial means
optimistic Deterministic bound only when a valid score_upper_bound is supplied
racing Confidence-bound statistical early stopping
none Full CV for every selected configuration

Important result attributes

  • best_params, best_score, best_state
  • completed_evaluations, partial_evaluations, history
  • n_evaluations, n_completed_evaluations, n_partial_evaluations
  • n_pruned, folds_saved, mean_fidelity
  • trust_expansions, trust_contractions, plateau_escapes
  • climbers_spawned, climbers_merged

Limitations

AGS still materializes the finite Cartesian product. Conditional spaces and lazy candidate generation are planned for v4.0. AGS is not exhaustive and does not guarantee the global optimum.

License

MIT © Mohammad Jawad Hasan

Metadata

Release files for adaptive-greedy-search 3.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for adaptive-greedy-search 3.6.0
File Size Uploaded Provenance Actions
adaptive_greedy_search-3.6.0.tar.gz 23.8 kB With provenance Details for adaptive_greedy_search-3.6.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for adaptive-greedy-search 3.6.0
File Interpreter ABI Platform Actions
adaptive_greedy_search-3.6.0-py3-none-any.whl Python 3 none any Details for adaptive_greedy_search-3.6.0-py3-none-any.whl

Total release size: 45.0 kB

Release files / adaptive_greedy_search-3.6.0.tar.gz

Download URL adaptive_greedy_search-3.6.0.tar.gz
Size 23.8 kB
Tags Source
SHA-256 checksum
How to use checksums
476abe4eb0ed05cbae1243c5947fd06f097035a120db88996c18970a24a9a1ec
BLAKE2b-256 checksum
How to use checksums
67d80bcf38ec42658531970b9d997271b692903a5fb54d9c0fb4ae02988a8361
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

Release files / adaptive_greedy_search-3.6.0-py3-none-any.whl

Download URL adaptive_greedy_search-3.6.0-py3-none-any.whl
Size 21.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
eaa41f2615e9e7405eb7b80a6cac49904361a13e8f2544f389a48cddae414170
BLAKE2b-256 checksum
How to use checksums
09db70415c86f412aa5fdf74788324b2161bade7b331bb949a4371bb27ef9696
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

Security for adaptive-greedy-search 3.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for adaptive_greedy_search-3.6.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 27, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page