Skip to main content

Adaptive Greedy Search (AGS)

Adaptive Greedy Search (AGS) v3.7.1 is a surrogate-assisted, multi-start adaptive local-search optimizer for finite hyperparameter spaces.

AGS combines typed and conditional search-space geometry, RF/GP/KDE surrogate guidance, adaptive trust-region climbers, confidence-bound multi-fidelity racing, diversity-aware batch search, and lazy candidate generation.

from ags import AdaptiveGreedySearch

What's new in 3.7.1 — Internal profiling

AGS can now expose where wall-clock time is spent inside the optimizer without requiring an external profiler. Profiling is opt-in so normal runs avoid timing overhead.

search = AdaptiveGreedySearch(
    estimator,
    param_grid,
    max_evaluations=60,
    profiling=True,
)

search.fit(X, y)
report = search.profile_report()

The report separates major internal phases such as proposal generation, local-neighborhood search, candidate-pool generation, acquisition scoring, surrogate fitting, diversity scoring, plateau escape, respawn, climber updates, and parallel evaluation-batch wall time. It also reports aggregate worker-side model-evaluation time.

Interpretation: deep timings are inclusive and may overlap. Worker compute is summed across workers and can exceed total wall time under parallel execution; it must not be subtracted directly from wall time.

What's new in 3.7.0

Conditional hyperparameters

AGS can now represent parameters that are active only for selected values of an earlier parent parameter.

from ags import (
    AdaptiveGreedySearch,
    CategoricalParameter,
    ConditionalParameter,
    IntegerParameter,
    NumericParameter,
)
from sklearn.svm import SVC

param_grid = {
    "kernel": CategoricalParameter(["linear", "rbf", "poly"]),
    "gamma": ConditionalParameter(
        NumericParameter([0.001, 0.01, 0.1, 1.0], scale="log"),
        parent="kernel",
        active_values=["rbf", "poly"],
    ),
    "degree": ConditionalParameter(
        IntegerParameter([2, 3, 4, 5]),
        parent="kernel",
        active_values=["poly"],
    ),
}

search = AdaptiveGreedySearch(
    SVC(),
    param_grid,
    max_evaluations=30,
)

For a state with kernel="linear", gamma and degree are inactive and are not passed to the estimator. For kernel="rbf", gamma is active but degree is omitted. For kernel="poly", both are active.

Conditional parents must currently appear before their children in param_grid insertion order.

Lazy candidate generation

AGS no longer constructs:

list(itertools.product(...))

for the entire search space.

Instead, global initialization, space-filling search, and restart logic operate on bounded candidate pools generated lazily:

search = AdaptiveGreedySearch(
    estimator,
    param_grid,
    candidate_pool_size=256,
)

This changes memory growth from being directly tied to the full Cartesian-product size to being tied primarily to the candidate pool, visited observations, and local neighborhoods.

search.search_space_size_upper_bound reports the raw Cartesian upper bound, while search.lazy_search indicates that the complete grid is not materialized.

Conditional-aware geometry and surrogate encoding

Inactive dimensions use an internal inactive state and an explicit activity mask in surrogate features.

Distance behaves as follows:

  • inactive in both states → no contribution;
  • active in one state but inactive in the other → activity mismatch penalty;
  • active in both → the wrapped parameter's normal numeric / ordinal / categorical distance.

Local neighbors are canonicalized after parent changes, so the search does not intentionally propose invalid conditional configurations.

Earlier 3.x improvements

v3.4 — Adaptive trust-region climbers

Each climber tracks its current state separately from its personal best and adapts trust_radius after success or failure.

v3.5 — Multi-fidelity confidence racing

pruning_strategy="racing" treats CV folds as fidelity and can stop statistically weak candidates before full CV.

v3.6 — Diversity-aware batch search and restart

Parallel proposals and respawns balance model quality with typed-distance diversity.

Installation

pip install adaptive-greedy-search

Quickstart

from ags import AdaptiveGreedySearch
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier

X, y = make_classification(
    n_samples=500,
    n_features=20,
    random_state=0,
)

param_grid = {
    "n_estimators": [50, 100, 150, 200],
    "max_depth": [3, 5, 7, 9, None],
    "min_samples_split": [2, 4, 6, 8],
}

search = AdaptiveGreedySearch(
    RandomForestClassifier(random_state=0),
    param_grid,
    cv=5,
    scoring="accuracy",
    max_evaluations=30,
    n_climbers=6,
    n_jobs=-1,
    surrogate_type="rf",
    acquisition="auto",
    pruning_strategy="racing",
    candidate_pool_size=256,
    random_state=42,
)

search.fit(X, y)

print("Best parameters:", search.best_params)
print("Best CV score:", search.best_score)
print("Evaluations:", search.n_evaluations)
print("Folds saved:", search.folds_saved)
print("Mean fidelity:", search.mean_fidelity)

Search-space API

Plain Python lists remain supported.

Explicit parameter types:

Type Purpose
NumericParameter Numeric values with linear or log geometry
FloatParameter Explicit real-valued finite grid
IntegerParameter Explicit integer finite grid
OrdinalParameter Ordered choices using rank distance
CategoricalParameter Nominal values using Hamming geometry
ConditionalParameter Wraps another parameter and activates it only for selected parent values

All values are still explicitly enumerated. AGS 3.7 does not turn these classes into continuous-domain optimizers.

Surrogates and acquisitions

Surrogate Meaning
rf Default. Random Forest mean + between-tree dispersion proxy
gp Gaussian Process predictive mean/std
kde Good/bad KDE density-ratio ranking
tpe Deprecated compatibility alias for kde; not a full TPE

Acquisition options are auto, ucb, ei / expected_improvement, and density_ratio where compatible.

Only completed full-CV evaluations are used as complete surrogate training targets.

Pruning / racing

Strategy Meaning
percentile Heuristic comparison with historical partial means
optimistic Safe only with a valid explicit score_upper_bound
racing Confidence-bound statistical early stopping
none Full CV for every selected configuration

Partial observations never become the reported winner.

Important 3.7 parameters and attributes

  • profiling=False: enable low-overhead internal phase timing when set to True.
  • profile_report(): return and optionally print the structured timing breakdown from the latest fit.
  • candidate_pool_size=256: maximum target size for lazily generated global candidate pools.
  • search_space_size_upper_bound: raw product-size upper bound.
  • lazy_search: True when using the structured lazy engine.
  • has_conditional_parameters: whether the configured search space contains conditionals.
  • best_params, best_score, best_state
  • completed_evaluations, partial_evaluations, history
  • folds_saved, mean_fidelity
  • trust_expansions, trust_contractions
  • climbers_spawned, climbers_merged, plateau_escapes

Current limitations

  • Search values are finite and explicitly listed.
  • Conditional dependencies are currently parent → child and the parent must appear earlier in the mapping.
  • Lazy candidate pools reduce full-grid memory pressure, but they do not guarantee exhaustive coverage of huge spaces.
  • AGS remains a heuristic optimizer and does not guarantee the global optimum.

License

MIT © Mohammad Jawad Hasan

Metadata

Release files for adaptive-greedy-search 3.7.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for adaptive-greedy-search 3.7.1
File Size Uploaded Provenance Actions
adaptive_greedy_search-3.7.1.tar.gz 29.4 kB With provenance Details for adaptive_greedy_search-3.7.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for adaptive-greedy-search 3.7.1
File Interpreter ABI Platform Actions
adaptive_greedy_search-3.7.1-py3-none-any.whl Python 3 none any Details for adaptive_greedy_search-3.7.1-py3-none-any.whl

Total release size: 54.8 kB

Release files / adaptive_greedy_search-3.7.1.tar.gz

Download URL adaptive_greedy_search-3.7.1.tar.gz
Size 29.4 kB
Tags Source
SHA-256 checksum
How to use checksums
57c9af06cf71b14e220328480933ca9eb9a83d8eb54a14093d32bac827e48c5c
BLAKE2b-256 checksum
How to use checksums
4ec4e959dbd9a52392ecb573f21f040571d8cdee5a395b515ba5c7ddc522c762
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.

Transparency log

Release files / adaptive_greedy_search-3.7.1-py3-none-any.whl

Download URL adaptive_greedy_search-3.7.1-py3-none-any.whl
Size 25.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
31720c15e985162e694ef96e2785f0d33dc7781d56c5f0951c67d47c685470a3
BLAKE2b-256 checksum
How to use checksums
c64644a781603226b6050dcdf4bf60ceb8fce60c1245d0d4eef7f172354ad9c7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.

Transparency log

Security for adaptive-greedy-search 3.7.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for adaptive_greedy_search-3.7.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page