In-context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization
Project description
In-context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization
This repository contains the official code for our ICML 2024 paper. This main
branch provides the Freeze-Thaw PFN surrogate (FT-PFN)
surrogate model as a drop-in surrogate for multi-fidelity Bayesian Optimization loops. Along with the synthetic prior generation and training code. To reproduce experiments from the above paper version, please refer to the branch icml-2024
.
To use the ifBO
algorithm in practice, please refer to NePS, a package for hyperparameter optimization that maintains the latest, improved ifBO
version (TBA, TODO).
Setup
conda create -n ifBO-env python=3.10 setuptools
conda activate ifBO-env
pip install -e .
Surrogate versions
Version | Identifier | Notes |
---|---|---|
0.0.1 | ICML '24 submission | FT-PFN from ifBO, trained on LCNet curves, DPL power law, broke scaling law |
Surrogate usage API
# To initialize the surrogate and load pretrained weights
from ifbo.surrogate import FTPFN
model = FTPFN()
NOTE: This creates a .model/
directory in the current working directory for the surrogate model. To have control over this specify a target_path: Path
when initializing.
To cite:
If using our surrogate, code, experiment setup, kindly cite using:
@inproceedings{
rakotoarison-icml24,
title={In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization},
author={H. Rakotoarison and S. Adriaensen and N. Mallik and S. Garibov and E. Bergman and F. Hutter},
booktitle={Forty-first International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=VyoY3Wh9Wd}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file ifbo-0.3.10.tar.gz
.
File metadata
- Download URL: ifbo-0.3.10.tar.gz
- Upload date:
- Size: 709.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d74a3212db16cfc9a682533ee97e0c44d032b38f55810b8c1d0ca7ba469f905 |
|
MD5 | 76c8a0d2f1d34a381cfb90ab9cb2be8c |
|
BLAKE2b-256 | cd0f12d90f00a567547e59d2f594acac7a96826192c32118df965e2d8f173543 |