kantun 0.0.3

Lightweight, model-agnostic hyperparameter tuning. Works standalone or alongside KANBoost.

Kantun

A lightweight, model-agnostic hyperparameter tuning library.

Built as a companion to KANBoost, but with zero hard dependency on it — Kantun works with any estimator that follows a scikit-learn-like fit/predict/predict_proba interface. Use it with KANBoost, with RandomForestClassifier, with your own custom model — your choice.

Why a separate package?

Not everyone using KANBoost needs hyperparameter search, and not everyone doing hyperparameter search needs KANBoost. Splitting them keeps each library's dependency footprint minimal and lets Kantun be useful on its own.

Install

pip install -r requirements.txt
pip install -e .

To also tune KANBoost models, install it separately:

pip install -e ../kanboost_project   # local, or: pip install kanboost (once published to PyPI)
# repo: https://github.com/tuamah/kanboost

Quickstart

from kantun import KantunSearch
from kanboost import KANBoostClassifier

param_space = {
    "n_estimators": [30, 60, 100],
    "learning_rate": [0.1, 0.2, 0.3],
    "kan_hidden": [3, 4, 6],
    "kan_grid": [2, 3],
}

search = KantunSearch(
    KANBoostClassifier,
    param_space,
    n_iter=10,
    cv=3,
    scoring="auc",
)
search.fit(X, y)

print(search.best_params_, search.best_score_)
best_model = search.best_estimator_          # ready to use
results_df = search.results_dataframe()       # sorted leaderboard

Works with any sklearn-style estimator, not just KANBoost

from sklearn.ensemble import RandomForestClassifier
from kantun import KantunSearch

search = KantunSearch(
    RandomForestClassifier,
    {"n_estimators": [50, 100], "max_depth": [3, 5, None]},
    n_iter=5, cv=3, scoring="f1",
    use_eval_set=False,   # RandomForestClassifier.fit() has no eval_set kwarg
)
search.fit(X, y)

How it decides whether to use early stopping

KantunSearch inspects the target model class's fit() signature. If it finds an eval_set parameter (as KANBoost's estimators do), it automatically passes eval_set=(X_val, y_val) on each fold so early stopping kicks in during the search itself. You can override this with use_eval_set=True/False explicitly.

Supported scoring

• Classification: "auc" (default), "f1", "accuracy"
• Regression: "neg_mse" (default), "neg_mae"

Search types

• search_type="random" (default): samples n_iter random combinations

• search_type="grid": tries every combination in param_distributions

• search_type="halving": successive halving -- starts every candidate on a small, stratified subsample of each fold's training data (held-out validation data is always full and untouched), keeps the top 1/halving_factor by score, and grows the training subsample by halving_factor each round until a round trains on the full data. Training-set size is the resource halved (not, say, n_estimators), since it's the only resource meaningful for any estimator -- kantun tunes arbitrary sklearn-compatible models, not just KANBoost. Useful when you have more candidates than you can afford to fully evaluate.

  search = KantunSearch(
      KANBoostClassifier, param_space, search_type="halving",
      n_iter=20, cv=3, halving_factor=3, min_resource=50,
  )
  

Speeding up an expensive search

Two independent knobs, useful together or separately, both aimed at the case kantun was built for: tuning an estimator that's slow to fit per combination (like KANBoost, ~10-20x a tree ensemble):

• n_jobs: evaluate multiple param combos concurrently (threads, not processes -- safe for CUDA device selection, and PyTorch releases the GIL during tensor ops so real overlap still happens).

  search = KantunSearch(KANBoostClassifier, param_space, n_jobs=4)
  

• prune=True: abandon a combo after its first CV fold if that fold's score already falls more than prune_margin standard deviations (of the current best combo's own fold spread) below the running best -- skips the remaining cv - 1 folds for combos that are essentially never going to become the best. A pruned combo's single-fold score is recorded in cv_results_ ("pruned": True) but never becomes best_params_/best_score_. Off by default; the first combo evaluated is never pruned (there's nothing to compare against yet).

  search = KantunSearch(KANBoostClassifier, param_space, prune=True, prune_margin=1.0)
  

License

MIT — see LICENSE.