DFO-LS 1.5.0

A flexible derivative-free solver for (bound constrained) nonlinear least-squares minimization

DFO-LS is a flexible package for solving nonlinear least-squares minimization, without requiring derivatives of the objective. It is particularly useful when evaluations of the objective function are expensive and/or noisy. DFO-LS is more flexible version of DFO-GN.

The main algorithm is described in our paper [1] below.

If you are interested in solving general optimization problems (without a least-squares structure), you may wish to try Py-BOBYQA, which has many of the same features as DFO-LS.

Documentation

See manual.pdf or here.

Citation

The development of DFO-LS is outlined over several publications:

1. C Cartis, J Fiala, B Marteau and L Roberts, Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers, ACM Transactions on Mathematical Software, 45:3 (2019), pp. 32:1-32:41 [preprint arXiv 1804.00154] .

2. M Hough and L Roberts, Model-Based Derivative-Free Methods for Convex-Constrained Optimization, SIAM Journal on Optimization, 21:4 (2022), pp. 2552-2579 [preprint arXiv 2111.05443].

3. Y Liu, K H Lam and L Roberts, Black-box Optimization Algorithms for Regularized Least-squares Problems, arXiv preprint arXiv:arXiv:2407.14915, 2024.

If you use DFO-LS in a paper, please cite [1]. If your problem has constraints, including bound constraints, please cite [1,2]. If your problem includes a regularizer, please cite [1,3].

Requirements

DFO-LS requires the following software to be installed:

• Python 3.9 or higher (http://www.python.org/)

Additionally, the following python packages should be installed (these will be installed automatically if using pip, see Installation using pip):

• NumPy (http://www.numpy.org/)

• SciPy version 1.11 or higher (http://www.scipy.org/)

• Pandas (http://pandas.pydata.org/)

Optional package: DFO-LS versions 1.2 and higher also support the trustregion package for fast trust-region subproblem solutions. To install this, make sure you have a Fortran compiler (e.g. gfortran) and NumPy installed, then run pip install trustregion. You do not have to have trustregion installed for DFO-LS to work, and it is not installed by default.

Installation using conda

DFO-LS can be directly installed in Anaconda environments using conda-forge:

$ conda install -c conda-forge dfo-ls

Installation using pip

For easy installation, use pip as root:

$ pip install DFO-LS

Note that if an older install of DFO-LS is present on your system you can use:

$ pip install --upgrade DFO-LS

to upgrade DFO-LS to the latest version.

Manual installation

Alternatively, you can download the source code from Github and unpack as follows:

$ git clone https://github.com/numericalalgorithmsgroup/dfols
$ cd dfols

DFO-LS is written in pure Python and requires no compilation. It can be installed using:

$ pip install .

To upgrade DFO-LS to the latest version, navigate to the top-level directory (i.e. the one containing pyproject.toml) and rerun the installation using pip, as above:

$ git pull
$ pip install .

Testing

If you installed DFO-LS manually, you can test your installation using the pytest package:

$ pip install pytest
$ python -m pytest --pyargs dfols

Alternatively, the HTML documentation provides some simple examples of how to run DFO-LS.

Examples

Examples of how to run DFO-LS are given in the documentation, and the examples directory in Github.

Uninstallation

If DFO-LS was installed using pip you can uninstall as follows:

$ pip uninstall DFO-LS

If DFO-LS was installed manually you have to remove the installed files by hand (located in your python site-packages directory).

Bugs

Please report any bugs using GitHub’s issue tracker.

License

This algorithm is released under the GNU GPL license. Please contact NAG for alternative licensing.