PyXAB 0.0.1

My short description for my project.

PyXAB - Python X-Armed Bandit

PyXAB is a Python open-source library for X-armed bandit, also known as continuous-arm bandit (CAB), Lipschitz bandit, global optimization (GO) and bandit-based blackbox optimization problems.

PyXAB includes implementations of different algorithms for X-armed bandit, such as HOO(Bubeck et al., 2011), HCT(Azar et al., 2014), and the most recent works GPO(Shang et al., 2019) and VHCT(Li et al, 2021). PyXAB also provides the most commonly-used synthetic objectives to evaluate the performance of different algorithms such as Garland and DoubleSine.

Quick Links

• Quick Example
• Documentations
• Features
  • X-armed bandit algorithms
  • Hierarchical partition
  • Synthetic objectives
• Contributing
• Citations

Quick Example

First define the blackbox objective, the parameter domain, the partition of the space, and the algorithm, e.g.

target = Garland()
domain = [[0, 1]]
partition = BinaryPartition
algo = T_HOO(rounds=1000, domain=domain, partition=partition)

At every round t, call algo.pull(t) to get a point. After receiving the (stochastic) reward for the point, call algo.receive_reward(t, reward) to give the algorithm the feedback

point = algo.pull(t)
reward = target.f(point) + np.random.uniform(-0.1, 0.1) # Uniform noise example
algo.receive_reward(t, reward)

Documentations

• The most up-to-date documentations for PyXAB

• The roadmap for our project

Features:

Stochastic X-armed bandit algorithms

• Algorithm starred are meta-algorithms (wrappers)

Algorithm	Research Paper	Year
T-HOO	X-Armed Bandit	2011
StoSOO	Stochastic Simultaneous Optimistic Optimization	2013
HCT	Online Stochastic Optimization Under Correlated Bandit Feedback	2014
POO*	Black-box optimization of noisy functions with unknown smoothness	2015
GPO*	General Parallel Optimization Without A Metric	2019
PCT	General Parallel Optimization Without A Metric	2019
VHCT	Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization	2021
VPCT	N.A. (GPO + VHCT)	N.A.

Hierarchical partition

Partition	Description
BinaryPartition	Equal-size binary partition of the parameter space, the split dimension is chosen uniform randomly
RandomBinaryPartition	The same as BinaryPartition but with a randomly chosen split point
DimensionBinaryPartition	Equal-size partition of the space with a binary split on each dimension, the number of children of one node is 2^d

Synthetic objectives

• Some of these objectives can be found here

Objectives	Mathematical Description	Image
Garland
DoubleSine
DifficultFunc
Ackley
Himmelblau
Rastrigin

Contributing

PyXAB is still under active development, and we appreciate all forms of help and contributions, including but not limited to

• Star and watch our project
• Open an issue for any bugs you find or features you want to add to our library
• Fork our project and submit a pull request with your valuable codes

Citations

If you use our package in your research or projects, we kindly ask you to cite our work

@article{li2021optimum,
  title={Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization},
  author={Li, Wenjie and Wang, Chi-Hua, Qifan Song and Cheng, Guang},
  journal={arXiv preprint arXiv:2106.09215},
  year={2021}
}