Hessian-free optimization for deep networks
Install the package via:
pip install hessianfree
To make sure things are working, open a python interpreter and enter:
import hessianfree as hf hf.demos.xor()
A simple xor training example will run, at the end of which it will display the target and actual outputs from the network.
Use this if you want to track the latest changes from the repository:
git clone https://github.com/drasmuss/hessianfree.git cd hessianfree python setup.py develop --user
(older versions may work, but are untested)
All the standard features of Hessian-free optimization from Martens (2010) and Martens and Sutskever (2011) are implemented (Gauss-Newton approximation, early termination, CG backtracking, Tikhonov damping, structural damping, etc.). In addition, the code has been designed to make it easy to customize the network you want to train, without having to modify the internal computations of the optimization process.
View the documentation at http://pythonhosted.org/hessianfree/
In addition, examples illustrating the main features of the code can be found in demos.py.