Skip to main content

In-context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization

Project description

arXiv

In-context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization

This repository contains the official code for our ICML 2024 paper. This main branch provides the Freeze-Thaw PFN surrogate (FT-PFN) surrogate model as a drop-in surrogate for multi-fidelity Bayesian Optimization loops. Along with the synthetic prior generation and training code. To reproduce experiments from the above paper version, please refer to the branch icml-2024.

To use the ifBO algorithm in practice, please refer to NePS, a package for hyperparameter optimization that maintains the latest, improved ifBO version (TBA, TODO).

Setup

conda create -n ifBO-env python=3.10 setuptools
conda activate ifBO-env
pip install -e .

Surrogate versions

Version Identifier Notes
0.0.1 ICML '24 submission FT-PFN from ifBO, trained on LCNet curves, DPL power law, broke scaling law

Surrogate usage API

# To initialize the surrogate and load pretrained weights
from ifbo.surrogate import FTPFN

model = FTPFN()

NOTE: This creates a .model/ directory in the current working directory for the surrogate model. To have control over this specify a target_path: Path when initializing.

To cite:

If using our surrogate, code, experiment setup, kindly cite using:

@inproceedings{
  rakotoarison-icml24,
  title={In-Context Freeze-Thaw Bayesian Optimization for Hyperparameter Optimization},
  author={H. Rakotoarison and S. Adriaensen and N. Mallik and S. Garibov and E. Bergman and F. Hutter},
  booktitle={Forty-first International Conference on Machine Learning},
  year={2024},
  url={https://openreview.net/forum?id=VyoY3Wh9Wd}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ifbo-0.3.10.tar.gz (709.0 kB view details)

Uploaded Source

File details

Details for the file ifbo-0.3.10.tar.gz.

File metadata

  • Download URL: ifbo-0.3.10.tar.gz
  • Upload date:
  • Size: 709.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.14

File hashes

Hashes for ifbo-0.3.10.tar.gz
Algorithm Hash digest
SHA256 2d74a3212db16cfc9a682533ee97e0c44d032b38f55810b8c1d0ca7ba469f905
MD5 76c8a0d2f1d34a381cfb90ab9cb2be8c
BLAKE2b-256 cd0f12d90f00a567547e59d2f594acac7a96826192c32118df965e2d8f173543

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page