Adaptive Greedy Search (AGS)
Adaptive Greedy Search (AGS) v3.7.1 is a surrogate-assisted, multi-start adaptive local-search optimizer for finite hyperparameter spaces.
AGS combines typed and conditional search-space geometry, RF/GP/KDE surrogate guidance, adaptive trust-region climbers, confidence-bound multi-fidelity racing, diversity-aware batch search, and lazy candidate generation.
from ags import AdaptiveGreedySearch
What's new in 3.7.1 — Internal profiling
AGS can now expose where wall-clock time is spent inside the optimizer without requiring an external profiler. Profiling is opt-in so normal runs avoid timing overhead.
search = AdaptiveGreedySearch(
estimator,
param_grid,
max_evaluations=60,
profiling=True,
)
search.fit(X, y)
report = search.profile_report()
The report separates major internal phases such as proposal generation, local-neighborhood search, candidate-pool generation, acquisition scoring, surrogate fitting, diversity scoring, plateau escape, respawn, climber updates, and parallel evaluation-batch wall time. It also reports aggregate worker-side model-evaluation time.
Interpretation: deep timings are inclusive and may overlap. Worker compute is summed across workers and can exceed total wall time under parallel execution; it must not be subtracted directly from wall time.
What's new in 3.7.0
Conditional hyperparameters
AGS can now represent parameters that are active only for selected values of an earlier parent parameter.
from ags import (
AdaptiveGreedySearch,
CategoricalParameter,
ConditionalParameter,
IntegerParameter,
NumericParameter,
)
from sklearn.svm import SVC
param_grid = {
"kernel": CategoricalParameter(["linear", "rbf", "poly"]),
"gamma": ConditionalParameter(
NumericParameter([0.001, 0.01, 0.1, 1.0], scale="log"),
parent="kernel",
active_values=["rbf", "poly"],
),
"degree": ConditionalParameter(
IntegerParameter([2, 3, 4, 5]),
parent="kernel",
active_values=["poly"],
),
}
search = AdaptiveGreedySearch(
SVC(),
param_grid,
max_evaluations=30,
)
For a state with kernel="linear", gamma and degree are inactive and are not passed to the estimator. For kernel="rbf", gamma is active but degree is omitted. For kernel="poly", both are active.
Conditional parents must currently appear before their children in param_grid insertion order.
Lazy candidate generation
AGS no longer constructs:
list(itertools.product(...))
for the entire search space.
Instead, global initialization, space-filling search, and restart logic operate on bounded candidate pools generated lazily:
search = AdaptiveGreedySearch(
estimator,
param_grid,
candidate_pool_size=256,
)
This changes memory growth from being directly tied to the full Cartesian-product size to being tied primarily to the candidate pool, visited observations, and local neighborhoods.
search.search_space_size_upper_bound reports the raw Cartesian upper bound, while search.lazy_search indicates that the complete grid is not materialized.
Conditional-aware geometry and surrogate encoding
Inactive dimensions use an internal inactive state and an explicit activity mask in surrogate features.
Distance behaves as follows:
- inactive in both states → no contribution;
- active in one state but inactive in the other → activity mismatch penalty;
- active in both → the wrapped parameter's normal numeric / ordinal / categorical distance.
Local neighbors are canonicalized after parent changes, so the search does not intentionally propose invalid conditional configurations.
Earlier 3.x improvements
v3.4 — Adaptive trust-region climbers
Each climber tracks its current state separately from its personal best and adapts trust_radius after success or failure.
v3.5 — Multi-fidelity confidence racing
pruning_strategy="racing" treats CV folds as fidelity and can stop statistically weak candidates before full CV.
v3.6 — Diversity-aware batch search and restart
Parallel proposals and respawns balance model quality with typed-distance diversity.
Installation
pip install adaptive-greedy-search
Quickstart
from ags import AdaptiveGreedySearch
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
X, y = make_classification(
n_samples=500,
n_features=20,
random_state=0,
)
param_grid = {
"n_estimators": [50, 100, 150, 200],
"max_depth": [3, 5, 7, 9, None],
"min_samples_split": [2, 4, 6, 8],
}
search = AdaptiveGreedySearch(
RandomForestClassifier(random_state=0),
param_grid,
cv=5,
scoring="accuracy",
max_evaluations=30,
n_climbers=6,
n_jobs=-1,
surrogate_type="rf",
acquisition="auto",
pruning_strategy="racing",
candidate_pool_size=256,
random_state=42,
)
search.fit(X, y)
print("Best parameters:", search.best_params)
print("Best CV score:", search.best_score)
print("Evaluations:", search.n_evaluations)
print("Folds saved:", search.folds_saved)
print("Mean fidelity:", search.mean_fidelity)
Search-space API
Plain Python lists remain supported.
Explicit parameter types:
| Type | Purpose |
|---|---|
NumericParameter |
Numeric values with linear or log geometry |
FloatParameter |
Explicit real-valued finite grid |
IntegerParameter |
Explicit integer finite grid |
OrdinalParameter |
Ordered choices using rank distance |
CategoricalParameter |
Nominal values using Hamming geometry |
ConditionalParameter |
Wraps another parameter and activates it only for selected parent values |
All values are still explicitly enumerated. AGS 3.7 does not turn these classes into continuous-domain optimizers.
Surrogates and acquisitions
| Surrogate | Meaning |
|---|---|
rf |
Default. Random Forest mean + between-tree dispersion proxy |
gp |
Gaussian Process predictive mean/std |
kde |
Good/bad KDE density-ratio ranking |
tpe |
Deprecated compatibility alias for kde; not a full TPE |
Acquisition options are auto, ucb, ei / expected_improvement, and density_ratio where compatible.
Only completed full-CV evaluations are used as complete surrogate training targets.
Pruning / racing
| Strategy | Meaning |
|---|---|
percentile |
Heuristic comparison with historical partial means |
optimistic |
Safe only with a valid explicit score_upper_bound |
racing |
Confidence-bound statistical early stopping |
none |
Full CV for every selected configuration |
Partial observations never become the reported winner.
Important 3.7 parameters and attributes
profiling=False: enable low-overhead internal phase timing when set toTrue.profile_report(): return and optionally print the structured timing breakdown from the latest fit.candidate_pool_size=256: maximum target size for lazily generated global candidate pools.search_space_size_upper_bound: raw product-size upper bound.lazy_search:Truewhen using the structured lazy engine.has_conditional_parameters: whether the configured search space contains conditionals.best_params,best_score,best_statecompleted_evaluations,partial_evaluations,historyfolds_saved,mean_fidelitytrust_expansions,trust_contractionsclimbers_spawned,climbers_merged,plateau_escapes
Current limitations
- Search values are finite and explicitly listed.
- Conditional dependencies are currently parent → child and the parent must appear earlier in the mapping.
- Lazy candidate pools reduce full-grid memory pressure, but they do not guarantee exhaustive coverage of huge spaces.
- AGS remains a heuristic optimizer and does not guarantee the global optimum.
License
MIT © Mohammad Jawad Hasan
Metadata
Release files for adaptive-greedy-search 3.7.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| adaptive_greedy_search-3.7.1.tar.gz | 29.4 kB | With provenance | Details for adaptive_greedy_search-3.7.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| adaptive_greedy_search-3.7.1-py3-none-any.whl | Python 3 | none | any | Details for adaptive_greedy_search-3.7.1-py3-none-any.whl |
Total release size: 54.8 kB
Release files / adaptive_greedy_search-3.7.1.tar.gz
| Download URL | adaptive_greedy_search-3.7.1.tar.gz |
|---|---|
| Size | 29.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
57c9af06cf71b14e220328480933ca9eb9a83d8eb54a14093d32bac827e48c5c
|
|
BLAKE2b-256 checksum How to use checksums |
4ec4e959dbd9a52392ecb573f21f040571d8cdee5a395b515ba5c7ddc522c762
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logRelease files / adaptive_greedy_search-3.7.1-py3-none-any.whl
| Download URL | adaptive_greedy_search-3.7.1-py3-none-any.whl |
|---|---|
| Size | 25.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
31720c15e985162e694ef96e2785f0d33dc7781d56c5f0951c67d47c685470a3
|
|
BLAKE2b-256 checksum How to use checksums |
c64644a781603226b6050dcdf4bf60ceb8fce60c1245d0d4eef7f172354ad9c7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logSecurity for adaptive-greedy-search 3.7.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for adaptive_greedy_search-3.7.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 29, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.