hypered 1.0.4

Simple hyper parameter tuning model.

Hypered: Hyperparameter Optimizer Library

This library provides a flexible interface for optimizing hyperparameters of any blackbox system. It implements bayesian optimization and supports the creation of various types of hyperparameter search spaces, such as real, integer, and categorical variables.

Features

• Hyperparameter Spaces: Define real, integer, and categorical variables.
• Objective Functions: Easily create minimization and maximization objectives.
• Experiment Management: Automatically handles experiment directories and parameter/result files.
• Parallel Execution: Supports parallel execution of experiments.

Installation

To install hypered, simply run:

pip install hypered

Usage

Step 1: Model Script

To use hypered you first need to define a model script that takes the hyper-parameters as an input json file and computes the loss/objective value and write it as a json file. Both input and output files should be provided by a command line arguments. The following is an example model script:

example.py

import argparse
import json
import numpy as np

def main(params_path: str, results_path: str):
    params = json.loads(open(params_path).read())

    op = params["vars"]["option"]
    x = params["vars"]["x"]

    if op == "first":
        loss = np.square(x - 5)
    elif op == "second":
        loss = np.abs(x - 3) - 2
    else:
        print("Invalid option", op)
        exit(0)

    print(x, loss)
    results = {
        "loss": loss
    }
    with open(results_path, "w") as f:
        f.write(json.dumps(results))

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Simple model.")
    parser.add_argument("params", type=str, help="Params file.")
    parser.add_argument("results", type=str, help="Results file.")
    args = parser.parse_args()
    main(args.params, args.results)

Step 2: Configuration File

Next we need to define a configuration file that specifies the hyper parameters as well as the objective function. Below is an example configuration file:

example.conf

optimize(
    name="learning_params",
    objective=minimize("loss"),
    binary="python3 example.py {params_path} {results_path}",
    random_starts=10,
    iterations=30,
    parallelism=8,
    params={
        "output_dir": experiment_dir(),
        "device_id": device_id(4),
        "vars": {
            "option": categorical(["first", "second"]),
            "x": real(-10, 10)
        }
    }
)

Step 3: Running the Hyperparameter Optimizer

To run the hyperparameter optimizer, use the hypered script with the path to your configuration file:

hypered example.conf

This will start the optimization process as defined in your configuration file and output the best parameters.

Step 4: Visualize the Results on Dashboard

Finally, you can visualize the results of hyper-parameter optimization on hypered-dash with the following command:

hypered-dash

Reference

Optimizers

optimize

This function performs hyperparameter optimization using Gaussian Processes.

Arguments:

• name (str): The name of the parameter group.
• objective (function): The objective function to minimize or maximize.
• binary (str): The command line binary to execute the experiment.
• params (dict): The dictionary of parameters to optimize.
• random_starts (int, optional): The number of random initialization points.
• iterations (int, optional): The number of iterations to run the optimization.
• seed (int, optional): The random seed for reproducibility.
• parallelism (int, optional): The number of parallel jobs to run.
• cwd (str, optional): The current working directory for the subprocess.

Note that you can use predefined variables {params_path} and {results_path} in your binary string to specify the path to parameters and results json files accordingly.

Objectives

minimize

Minimize a given variable.

maximize

Maximize a given variable.

Variables

uniform

Returns the string identifier for a uniform distribution.

log_uniform

Returns the string identifier for a log-uniform distribution.

real

Class for defining a real-valued hyperparameter.

integer

Class for defining an integer-valued hyperparameter.

categorical

Class for defining a categorical hyperparameter.

Utilities

experiment_dir

Retrieves the experiment directory from the context.

params_path

Retrieves the parameters path from the context.

results_path

Retrieves the results path from the context.

device_id

Returns a device ID in a round-robin fashion.

License

This library is licensed under the MIT License. See the LICENSE file for more details.

Contributing

Contributions are welcome! Please open an issue or submit a pull request on GitHub.

Contact

For any questions or issues, please open an issue on the GitHub repository.