How it works

GPry uses a Gaussian Process (GP) to create an interpolating model of the log-posterior density function, using as few evaluations as possible. It achieves that using active learning: starting from a minimal set of training samples, the next ones are chosen so that they maximise the information gained on the posterior shape. For more details, see section How GPry works of the documentation, and check out the GPry papers (see below).

GPry introduces some innovations with respect to previous similar approaches:

• It imposes weakly-informative priors on the target function, based on a comparison with an n-dimensional Gaussian, and uses that information e.g. for convergence metrics, balancing exploration vs. exploitation, etc.

• It introduces a parallelizable batch acquisition algorithm (NORA) which increases robustness, reduces overhead and enables the evaluation of the likelihood/posterior in parallel using multiple cores.

• Complementing the GP model, it implements an SVM classifier that learns the shape of uninteresting regions, where proposals are discarded, wherever the value of the likelihood is very-low (for increased efficiency) or undefined (for increased robustness).

At the moment, GPry utilizes a modification of the CPU-based scikit-learn GP implementation.

What kinds of likelihoods/posteriors should work with GPry?

• Non-stochastic log-probability density functions, smooth up to a small amount of (deterministic) numerical noise (less than 0.1 in log posterior).

• Log-likelihoods with large evaluation times, so that the GPry overhead is subdominant with respect to posterior evaluation. How slow depends on the number of dimensions and expected shape of the posterior distribution but as a rule of thumb, if an MCMC takes longer to converge than you’re willing to wait, you should give it a shot.

• The parameter space needs to be low-dimensional (less than 20 as a rule of thumb). In higher dimensions you might still gain considerable improvements in speed if your likelihood is sufficiently slow but the computational overhead of the algorithm increases considerably.

What may not work so well:

• Highly multimodal posteriors, especially if the separation between modes is large.

• Highly non-Gaussian posteriors, that would not be well modelled by orthogonal constant correlation lengths.

GPry is under active development, in order to mitigate some of those issues, so look out for new versions!

It does not work!

Please check out the Strategy and Troubleshooting page, or get in touch for issues or more general discussions.

What to cite

If you use GPry, please cite the following papers:

• arXiv:2211.02045 for the core algorithm.

• arXiv:2305.19267 for the NORA Nested-Sampling acquisition engine.

Some papers using GPry

• arXiv:2501.11320

• arXiv:2503.21871