Skip to main content

Package for tuning hyperparameters with resampling methods

Project description

Hyperparameter TuneRs

tuneRs is a small package for tuning hyperparameters using resampling methods instead of normal crossvalidation. Estimating model accuracy using resampling methods is much quicker that using k-fold crossvalidation--although resampling tends to underestimate accuracy more that crossvalidation. Resampling underestimates accuracy in a consistent fashion, however, which still makes it valuable for tuning hyperparameters. Due to it's consistency, choosing hyperparameters based on aggregated samples still gets within the neighborhood of maximal while being much, much faster. This is a package to help you get there.

GridSearchResample uses the grid search method to optimize hyperparameters.

RandomSearchResample uses the random search method to optimize hyperparameters.

Current Version is v0.55

This package is currently in the beginning stages and is very bare-bones


Use the package manager pip to install tuneRs.

pip install tuneRs


Both classes are meant to mimic the scikit-learn tuners (to a certain degree). A simple example would be:

import tuneRs

model = SVC(kernel='rbf')

parameters = {["gamma": [0.001, 0.01, 0.1, 1.0, 10.0],
		"C": [0.01, 0.1, 1.0, 10.0, 100.0, 1000.0]}

# Set up for a random hyperparameter search
tuner = tuneRs.RandomSearchResample(model, parameters, num_iters=300, sample_size=0.3, num_samples=12)

# Fit the tuner, y_train)

# Display the best parameters found

# Display the aggregate resample score of the best parameters <br/>

# Define our new model
model = tuner.best_estimator_

# Plot the resample accuracy distribution for the model with best hyperparameters <br/>

Future Plans

Multiple tuners are currently planned to be added. The next one will be a Bayesian search method. A dynamic version of grid search and random search is currently being worked on that iterates fit multiple times on increasingly small areas of the data space.



Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tuneRs-0.56.tar.gz (3.8 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page