Solves automatic numerical differentiation problems in one or more variables.

## numdifftools

The numdifftools library is a suite of tools written in _Python to solve automatic numerical differentiation problems in one or more variables. Finite differences are used in an adaptive manner, coupled with a Richardson extrapolation methodology to provide a maximally accurate result. The user can configure many options like; changing the order of the method or the extrapolation, even allowing the user to specify whether complex-step, central, forward or backward differences are used.

The methods provided are:

• Derivative: Compute the derivatives of order 1 through 10 on any scalar function.

• directionaldiff: Compute directional derivative of a function of n variables

• Gradient: Compute the gradient vector of a scalar function of one or more variables.

• Jacobian: Compute the Jacobian matrix of a vector valued function of one or more variables.

• Hessian: Compute the Hessian matrix of all 2nd partial derivatives of a scalar function of one or more variables.

• Hessdiag: Compute only the diagonal elements of the Hessian matrix

All of these methods also produce error estimates on the result.

Numdifftools also provide an easy to use interface to derivatives calculated with in _AlgoPy. Algopy stands for Algorithmic Differentiation in Python. The purpose of AlgoPy is the evaluation of higher-order derivatives in the forward and reverse mode of Algorithmic Differentiation (AD) of functions that are implemented as Python programs.

### Getting Started

Visualize high order derivatives of the tanh function

>>> import numpy as np
>>> import numdifftools as nd
>>> import matplotlib.pyplot as plt
>>> x = np.linspace(-2, 2, 100)
>>> for i in range(10):
...    df = nd.Derivative(np.tanh, n=i)
...    y = df(x)
...    h = plt.plot(x, y/np.abs(y).max())


plt.show()

Compute 1’st and 2’nd derivative of exp(x), at x == 1:

>>> fd = nd.Derivative(np.exp)        # 1'st derivative
>>> fdd = nd.Derivative(np.exp, n=2)  # 2'nd derivative
>>> np.allclose(fd(1), 2.7182818284590424)
True
>>> np.allclose(fdd(1), 2.7182818284590424)
True

Nonlinear least squares:

>>> xdata = np.reshape(np.arange(0,1,0.1),(-1,1))
>>> ydata = 1+2*np.exp(0.75*xdata)
>>> fun = lambda c: (c[0]+c[1]*np.exp(c[2]*xdata) - ydata)**2
>>> Jfun = nd.Jacobian(fun)
>>> np.allclose(np.abs(Jfun([1,2,0.75])), 0) # should be numerically zero
True

>>> fun = lambda x: np.sum(x**2)
>>> np.allclose(dfun([1,2,3]), [ 2.,  4.,  6.])
True

Compute the same with the easy to use interface to AlgoPy:

>>> import numdifftools.nd_algopy as nda
>>> import numpy as np
>>> fd = nda.Derivative(np.exp)        # 1'st derivative
>>> fdd = nda.Derivative(np.exp, n=2)  # 2'nd derivative
>>> np.allclose(fd(1), 2.7182818284590424)
True
>>> np.allclose(fdd(1), 2.7182818284590424)
True

Nonlinear least squares:

>>> xdata = np.reshape(np.arange(0,1,0.1),(-1,1))
>>> ydata = 1+2*np.exp(0.75*xdata)
>>> fun = lambda c: (c[0]+c[1]*np.exp(c[2]*xdata) - ydata)**2
>>> Jfun = nda.Jacobian(fun, method='reverse')
>>> np.allclose(np.abs(Jfun([1,2,0.75])), 0) # should be numerically zero
True

>>> fun = lambda x: np.sum(x**2)
>>> np.allclose(dfun([1,2,3]), [ 2.,  4.,  6.])
True

scipy.misc.derivative

### Documentation and code

Numdifftools works on Python 2.7+ and Python 3.0+.

Official releases available at: http://pypi.python.org/pypi/numdifftools

Bleeding edge: https://github.com/pbrod/numdifftools.

### Installation

If you have pip installed, then simply type:

\$ pip install numdifftools

to get the lastest stable version. Using pip also has the advantage that all requirements are automatically installed.

### Unit tests

To test if the toolbox is working paste the following in an interactive python session:

import numdifftools as nd
nd.test('--doctest-modules', '--disable-warnings')

### Acknowledgement

The numdifftools package for Python was written by Per A. Brodtkorb based on the adaptive numerical differentiation toolbox written in Matlab by John D’Errico [DErrico2006].

Numdifftools has as of version 0.9 been extended with some of the functionality found in the statsmodels.tools.numdiff module written by Josef Perktold [Perktold2014] and in the project report of [Verheyleweghen2014].

### References

[Lantoine2010]

Gregory Lantoine (2010), A methodology for robust optimization of low-thrust trajectories in multi-body environments, Phd thesis, Georgia Institute of Technology

[LantoineEtal2012]

Gregory Lantoine, R.P. Russell, and T. Dargent (2012) Using multicomplex variables for automatic computation of high-order derivatives, ACM Transactions on Mathematical Software, Vol. 38, No. 3, Article 16, April 2012, 21 pages, http://doi.acm.org/10.1145/2168773.2168774

[Luna-ElizarrarasEtal2012]

M.E. Luna-Elizarraras, M. Shapiro, D.C. Struppa1, A. Vajiac (2012), Bicomplex Numbers and Their Elementary Functions, CUBO A Mathematical Journal, Vol. 14, No 2, (61-80). June 2012.

Adriaen Verheyleweghen, (2014) Computation of higher-order derivatives using the multi-complex step method, Project report, NTNU

## Changelog

### Version 0.9.39 Jun 10, 2019

Robert Parini (1):
• Fix issue #43: numpy future warning

### Version 0.9.38 Jun 10, 2019

Andrew Nelson (1):
• MAINT: special.factorial instead of misc.factorial

Dougal J. Sutherland (1):

Per A Brodtkorb (140):
• Adjusted runtime for hypothesis tests to avoid failure and fixed pep8 failures.

• Fixed a bug in setup.cfg

• Replaced valarray function with numpy.full in step_generators.py

• Added try except on import of algopy

• Replaced numpy.testing.Tester with pytest.

• Removed dependence on pyscaffold.

• Simplified setup.py and setup.cfg

• Updated .travis.yml configuration.

• Reorganized the documentation.

• Ongoing work to simplify the classes.

• Replaced unittest with pytest.

• replaced , with .

• Reverted to coverage=4.3.4

• New attempt

• Fixed conflicting import

• Missing import of EPS

• Added missing FD_RULES = {}

• Removed pinned coverage, removed dependence on pyscaffold

• Updated .travis.yml and .appveyor.yml

• Replaced conda channel omnia with conda-forge

• Removed commented out code. Set pyqt=5 in appveyor.yml

• Updated codeclimate checks

• Dropped support for python 3.3 and 3.4. Added support for python 3.6, 3.7

• Simplified code.

• Pinned IPython==5.0 in order to make the testserver not crash.

• Removed line_profiler from requirements.txt

• Fix issue #37: Unable to install on Python 2.7

• Skip test_profile_numdifftools_profile_hessian and TestDoProfile

• Added missing import of warnings

• Added tests for the scripts from profile_numdifftools.py, _find_default_scale.py and run_benchmark.py.

• misssing import of warnings fixed.

• Renamed test so it comes last, because I suspect this test mess up the coverage stats.

• Reordered the tests.

• Cleaned up _find_default_scale.py

• Reverted: install of cython and pip install setuptools

• Disabled sonar-scanner -X for python 3.5 because it crashes.

• Reverted [options.packages.find] to exclude tests again

• Added cython and reverted to pip install setuptools

• Updated sphinx to 1.6.7

• Try to install setuptools with conda instead.

• Set version of setuptools==37.0

• Restricted sphinx in the hope that the docs will be generated.

• Removed exclusion of tests/ directory from test coverage.

• Refactored and removed commented out code.

• Fixed a bug in the docstring example: Made sure the shape passed on to zeros is an integer.

• Fixed c_abs so it works with algopy on python 3.6.

• Fixed flaky test and made it more robust.

• Fixed bug in .travis.yml

• Refactored the taylor function into the Taylor class in order to simplify the code.

• Fixed issue #35 and added tests

• Attempt to simplify complexity

• Updated project path

• Changed install of algopy

• Fixed small bugs

• Updated docstrings

• Changed Example and Reference to Examples and References in docstrings to comply with numpydoc-style.

• Renamed CHANGES.rst to CHANGELOG.rst

• Renamed source path

• Hack due to a bug in algopy or changed behaviour.

• Small fix.

• Try to reduce complexity

• Reduced cognitive complexity of min_num_steps

• Simplified code in Jacobian

• Merge branch ‘master’ of https://github.com/pbrod/numdifftools

• Fixed issue #34 Licence clarification.

• Locked coverage=4.3.4 due to a bug in coverage that cause code-climate test-reporter to fail.

• Added script for finding default scale

• updated from sonarcube to sonarcloud

• Made sure shape is an integer.

• Refactored make_step_generator into a step property

• Issue warning message to the user when setting the order to something different than 1 or 2 in Hessian.

• Reverted –timid option to coverage because it took too long time to run.

• Reverted –pep8 option

• pep8 + added –timid in .travis.yml coverage run in order to to increase missed coverage.

• Refactored taylor to reduce complexity

• No support for python 3.3. Added python 3.6

• Fixed a small bug and updated test.

• Removed unneccasarry perenthesis. Reduced the complexity of do_profile

• Removed assert False

• updated doctest in nd_scipy.py and profiletools.py install line_profiler on travis

• Updated tests

• Added test_profiletools.py and capture_stdout_and_stderr in testing.py

• Optimized numdifftools.core.py for speed: fd_rules are now only computed once.

• Only keeping html docs in the distribution.

• Added doctest and updated .pylintrc and requirements.txt

• Reduced time footprint on tests in the hope that it will pass on Travis CI.

• Prefer static methods over instance methods

• Better memory handling: This fixes issue #27

• Simplified input

• Merge branch ‘master’ of https://github.com/pbrod/numdifftools

• Updated link to the documentation.

Robert Parini (4):
• Avoid RuntimeWarning in _get_logn

• Allow fd_derivative to take complex valued functions

solarjoe (1):

### Version 0.9.20, Jan 11, 2017

Per A Brodtkorb (1):
• Updated the author email address in order for twine to be able to upload to pypi.

### Version 0.9.19, Jan 11, 2017

Per A Brodtkorb (1):
• Updated setup.py in a attempt to get upload to pypi working again.

### Version 0.9.18, Jan 11, 2017

Per A Brodtkorb (38):
• Updated setup

• Added more rigorous tests using hypothesis.

• Forced to use wxagg backend

• Moved import of matplotlib.pyplot to main in order to avoid import error on travis.

• Updated references.

• Attempt to automate sonarcube analysis

• Added testcoverage to sonarqube and codeclimate

• Simplified code

• Major change in api: class member variable self.f changed to self.fun

• Fixes issue #25 (Jacobian broken since 0.9.15)

### Version 0.9.17, Sep 8, 2016

Andrew Fowlie (1):

Per A Brodtkorb (79):

• Removed obsolete docs from core.py

• Updated appveyor.yml

• Simplified code

• Removed plot in help header

• Added Programming Language :: Python :: 3.5

• Simplified code

• Renamed bicomplex to Bicomplex

• Simplified example_functions.py

• Moved MinStepGenerator, MaxStepGeneretor and MinMaxStepGenerator to step_generators.py
• Unified the step generators

• Moved step_generator tests to test_step_generators.py

• Major simplification of step_generators.py

• Removed duplicated code + pep8

• Moved fornberg_weights to fornberg.py + added taylor and derivative

• Fixed print statement

• Replace xrange with range

• Made ‘backward’ and alias for ‘reverse’ in nd_algopy.py

• Expanded the tests + added test_docstrings to testing.py

• Replace string interpolation with format()

• Removed obsolete parameter

• Smaller start radius for Fornberg method

• Simplified “n” and “order” properties

• Simplified default_scale

• Removed unecessary parenthesis and code. pep8

• Fixed a bug in Dea + small refactorings.

• Avoid mutable default args and prefer static methods over instance-meth.

• Refactored to reduce cyclomatic complexity

• Changed some instance methods to static methods

• Renamed non-pythonic variable names

• Turned on xvfb (X Virtual Framebuffer) to imitate a display.

• Added extra test for Jacobian

• Replace lambda function with a def

• Removed unused import

• Fixed test_scalar_to_vector

• Updated test_docstrings

### Version 0.9.15, May 10, 2016

Cody (2):
• Migrated % string formating

• Migrated % string formating

Per A Brodtkorb (28):

• Replaced instance methods with static methods +pep8

• Merge branch ‘master’ of https://github.com/pbrod/numdifftools

• Fixed a bug: replaced missing triple quote

• Fixed failing tests

• Changed instance methods to static methods

• Made untyped exception handlers specific

• Replaced local function with a static method

• Simplified tests

• Removed duplicated code Simplified _Derivative._get_function_name

• exclude tests from testclimate

• Renamed test_functions.py to example_functions.py Added test_example_functions.py

Per A. Brodtkorb (2):
• Merge pull request #17 from pbrod/autofix/wrapped2_to3_fix

• Merge pull request #18 from pbrod/autofix/wrapped2_to3_fix-0

pbrod (17):
• updated conf.py

• added numpydoc>=0.5, sphinx_rtd_theme>=0.1.7 to setup_requires if sphinx

• updated setup.py

• Updated README.rst with info about how to install it using conda in an anaconda package.

• updated conda install description

• Fixed number of arguments so it does not differs from overridden ‘_default_base_step’ method

• Attempt to remove coverage of test-files

• Added directionaldiff function in order to calculate directional derivatives. Fixes issue #16. Also added supporting tests and examples to the documentation.

• Fixed isssue #19 multiple observations mishandled in Jacobian

• Moved rosen function into numdifftools.testing.py

• updated import of rosen function from numdifftools.testing

• Simplified code + pep8 + added TestResidue

• Updated readme.rst and replaced string interpolation with format()

• Cleaned Dea class + pep8

• Updated references for Wynn extrapolation method.

### Version 0.9.14, November 10, 2015

pbrod (53):
• Updated documentation of setup.py

• updated version

• Updated example

• Added .landscape.yml updated .coveragerc, .travis.yml

• updated docs/index.rst

• Removed unused code and added tests/test_extrapolation.py

• updated tests

• Removed dependence on wheel, numpydoc>=0.5 and sphinx_rtd_theme>=0.1.7 (only needed for building documentation)

• updated conda path in .travis.yml

• added omnia channel to .travis.yml

• Added conda_recipe files Filtered out warnings in limits.py

### Version 0.9.13, October 30, 2015

pbrod (21):

• updated Limits.

• Made it possible to differentiate complex functions and allow zero’th order derivative.

• Updated structure in setup according to pyscaffold version 2.4.2.

• Updated setup.cfg and deleted duplicate tests folder.

• removed unused code.

• Added required appveyor install scripts

• Fixed bug in appveyor.yml.

• updated appveyor.yml.

• Removed import matplotlib.

Justin Lecher (1):
• Fix min version for numpy.

kikocorreoso (1):
• fix some prints on run_benchmark.py to make it work with py3

### Version 0.9.12, August 28, 2015

pbrod (12):

• Updated documentation.

• Updated version in conf.py.

• Updated CHANGES.rst.

• Reimplemented outlier detection and made it more robust.

• Updated main tests folder.

• Moved Richardson and dea3 to extrapolation.py.

• Making a new release in order to upload to pypi.

### Version 0.9.11, August 27, 2015

pbrod (2):
• Fixed sphinx-build and updated docs.

• Fixed issue #9 Backward differentiation method fails with additional parameters.

### Version 0.9.10, August 26, 2015

pbrod (7):
• Fixed sphinx-build and updated docs.

• Added more tests to nd_algopy.

• Dropped support for Python 2.6.

### Version 0.9.4, August 26, 2015

pbrod (7):
• Fixed sphinx-build and updated docs.

### Version 0.9.3, August 23, 2015

Paul Kienzle (1):
• more useful benchmark plots.

pbrod (7):
• Fixed bugs and updated docs.

• Major rewrite of the easy to use interface to Algopy.

• Added possibility to calculate n’th order derivative not just for n=1 in nd_algopy.

• Added tests to the easy to use interface to algopy.

### Version 0.9.2, August 20, 2015

pbrod (3):
• Updated documentation

• Added parenthesis to a call to the print function

• Made the test less strict in order to pass the tests on Travis for python 2.6 and 3.2.

### Version 0.9.1, August 20,2015

Christoph Deil (1):
• Fix Sphinx build

pbrod (47):
• Total remake of numdifftools with slightly different call syntax.
• Can compute derivatives of order up to 10-14 depending on function and method used.

• Updated documentation and tests accordingly.

• Fixed a bug in dea3.

• Added bicomplex class for testing the complex step second derivative.

• Added fornberg_weights_all for computing optimal finite difference rules in a stable way.

• Added higher order complex step derivative methods.

### Version 0.7.7, December 18, 2014

pbrod (35):
• Got travis-ci working in order to run the tests automatically.

• Fixed bugs in Dea class.

• Fixed better error estimate for the Hessian.

• Fixed tests for python 2.6.

• Restructerd folders of numdifftools.

### Version 0.7.3, December 17, 2014

pbrod (5):
• Small cosmetic fixes.

• pep8 + some refactorings.

• Simplified code by refactoring.

### Version 0.6.0, February 8, 2014

pbrod (20):

• Simplified call to Derivative: removed step_fix.

• Deleted unused code.

• Simplified and Refactored. Now possible to choose step_num=1.

• Changed default step_nom from max(abs(x0), 0.2) to max(log2(abs(x0)), 0.2).

• pep8ified code and made sure that all tests pass.

### Version 0.5.0, January 10, 2014

pbrod (9):
• Updated the examples in Gradient class and in info.py.

• Added test for vec2mat and docstrings + cosmetic fixes.

• Refactored code into private methods.

• Fixed issue #7: Derivative(fun)(numpy.ones((10,5)) * 2) failed.

• Made print statements compatible with python 3.

### Version 0.4.0, May 5, 2012

pbrod (1)
• Fixed a bug for inf and nan values.

### Version 0.3.5, May 19, 2011

pbrod (1)
• Fixed a bug for inf and nan values.

### Version 0.3.4, Feb 24, 2011

pbrod (11)
• Made automatic choice for the stepsize more robust.

• Added easy to use interface to the algopy and scientificpython modules.

### Version 0.3.1, May 20, 2009

pbrod (4)
• First version of numdifftools published on google.code

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

• Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

• Neither the name of the copyright holders nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

## Project details

Uploaded Source
Uploaded Python 2 Python 3