Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Package for tuning hyperparameters with resampling methods

Project description

Hyperparameter TuneRs

tuneRs is a small package for tuning hyperparameters using resampling methods instead of normal crossvalidation. Estimating model accuracy using resampling methods is much quicker that using k-fold crossvalidation--although resampling tends to underestimate accuracy more that crossvalidation. Resampling underestimates accuracy in a consistent fashion, however, which still makes it valuable for tuning hyperparameters. Due to it's consistency, choosing hyperparameters based on aggregated samples still gets within the neighborhood of maximal while being much, much faster. This is a package to help you get there.

GridSearchResample uses the grid search method to optimize hyperparameters.

RandomSearchResample uses the random search method to optimize hyperparameters.

Current Version is v0.57

This package is currently in the beginning stages and is very bare-bones

Installation

Use the package manager pip to install tuneRs.

pip install tuneRs

Usage

Both classes are meant to mimic the scikit-learn tuners (to a certain degree). A simple example would be:

import tuneRs

model = SVC(kernel='rbf')

parameters = {["gamma": [0.001, 0.01, 0.1, 1.0, 10.0],
		"C": [0.01, 0.1, 1.0, 10.0, 100.0, 1000.0]}

# Set up for a random hyperparameter search
tuner = tuneRs.RandomSearchResample(model, parameters, num_iters=300, sample_size=0.3, num_samples=12)

# Fit the tuner
tuner.fit(X_train, y_train)

# Display the best parameters found
tuner.best_params_

# Display the aggregate resample score of the best parameters <br/>
tuner.best_score_

# Define our new model
model = tuner.best_estimator_

# Plot the resample accuracy distribution for the model with best hyperparameters <br/>
tuner.plot_best()

Future Plans

Multiple tuners are currently planned to be added. The next one will be a Bayesian search method. A dynamic version of grid search and random search is currently being worked on that iterates fit multiple times on increasingly small areas of the data space.

License

Lol

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for tuneRs, version 0.57
Filename, size File type Python version Upload date Hashes
Filename, size tuneRs-0.57.tar.gz (6.2 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page