topsearch 0.0.1

A Python package for topographical analysis of machine learning models and physical systems

TopSearch

This project is currently migrating to open source. Please check back regularly as more functionality is being added and updated

Introduction

The TopSearch package provides functionality to map the topography of surfaces, and can be used to map the solution space of machine learning algorithms. Many machine learning algorithms have non-convex loss (or cost) functions, and the aim of fitting is usually to locate low-valued or diverse solutions. An understanding of the solution space organisation provides key understanding about the reproducibility, explainability and performance of ML methods.

The methodology derives from the field of chemical physics, and is agnostic to the given surface, allowing application to a wide range of machine learning algorithms. Leveraging ideas from chemical physics we can assess the performance and reliability of neural networks, Gaussian processes, Bayesian optimisation, clustering algorithms and understand the effect of dataset roughness on model performance. Application of the same framework to many different machine learning paradigms provides a route for transferable understanding, and its application to machine learning is reviewed in this paper.

Overview

The topographical mapping is performed using the energy landscape framework. The energy landscape framework, developed in chemical physics, encodes surfaces as a network of stationary points. These stationary points, points with zero gradient, can be either local minima or transition states. Transition states are maximal in a single direction and local minimisation along that given direction (both forwards and backwards) locates two connected minima. Each transition state gives the lowest barrier between its two connected minima, and provides information about the intermediate behaviour of the function. In the network, each minimum is a node and edges exist between minima connected by a transition state. The complete network of minima and transition states constitutes the solution landscape, and we show an example landscape below.

Top: A contour plot of the original surface with the stationary points, and their connections, overlaid. The minima are given in green, the transition states in red, and the connections between them with solid black lines. Bottom: The corresponding network abstraction of the surface. Here, the separation between connected nodes is specified by the height of the transition state between them.

Topographical mapping of surfaces involves two main steps:

• Global optimisation → location of the global minimum and other low-valued local minima of the surface
• Landscape exploration → attempt transition state searches between pairs of minima to generate a fully connected set of minima

Global optimisation is usually performed using the basin-hopping algorithm within TopSearch. Basin-hopping is a modified Monte Carlo approach that includes local minimisation at each step to efficiently optimise complex solution spaces. During global optimisation we store all unique minima that we encounter, which gives us the initial set of minima that will be connected by transition state searches.

Landscape exploration involves selecting pairs of minima and attempting to find transition states between them. Transition state location is usually performed using a combination of double-ended and single-ended transition state searches. Double-ended searches aim to locate the lowest-valued path between a given pair of minima, and we use the nudged elastic band algorithm within TopSearch. Single-ended methods start from a single point and follow the lowest eigenmode towards the nearest transition state, and this is performed using hybrid eigenvector-following. The nudged elastic band algorithm locates an approximate minimum energy path between two chosen minima, and the maxima on this path are refined to true transition states using hybrid eigenvector-following. There are a variety of schemes to decide which pairs of minima should be selected for transition state connections, all of which aim to produce a fully connected network of minima and explore important regions of solution space.

For more details of the methodology please refer to common_citations.md.

Installation

Instructions are for installation into a conda environment. First create the conda environment

conda create -n topsearch_env python
conda activate topsearch_env

The package is available on PyPI and can be installed using pip

pip install topsearch

or directly from the git source using

pip install git+https://github.com/IBM/topography-searcher

For those not importing the package directly you can set up the proper enivornment from the pyproject.toml file with

poetry install

We can test the environment build by running

pytest

within the tests directory. For a successful build we should have all tests pass. By default we do not test for the molecular_potentials as the dependencies are not shipped by default due to the large size. If this is the case then enjoy using TopSearch!

Examples

We provide several examples to illustrate the tasks that TopSearch can perform in examples. These examples are provided as both an annotated Jupyter notebook or a python script (the scripts are further separated for ease), and each example has a detailed description of its content within the README. example_function as a notebook or scripts are the best place to start for an introduction to the methodology, where we apply it to some simple test functions.

Contributors

This package is written and maintained by Luke Dicks at IBM Research as part of the AI-Enriched Simulation team. Please contact Luke ([e-mail]) or Edward Pyzer-Knapp ([e-mail]) for questions about how to use and/or contribute.

License

TopSearch is an open-source software licensed under the MIT License. Check the details in the LICENSE file.

Citations

If you use this package please cite it appropriately using the 'Cite this repository' dropdown in the right sidebar. Moreover, we also provide a bibliography of previous relevant work in common_citations.md. This file provides references to energy landscape algorithms and their applications to different fields of machine learning, each given as a bibtex entry with the handle providing a summary of the content.