PyTorch package for all things neural differential equations.
Project description
Torchdyn is a PyTorch library dedicated to numerical deep learning: differential equations, integral transforms, numerical methods. Maintained by DiffEqML.
Quick Start
Torchdyn provides utilities and layers to easily construct numerical deep learning models. For example, neural differential equations:
from torchdyn.core import NeuralODE
# your preferred torch.nn.Module here
f = nn.Sequential(nn.Conv2d(1, 32, 3),
nn.Softplus(),
nn.Conv2d(32, 1, 3)
)
nde = NeuralODE(f)
And you have a trainable model. Feel free to combine Torchdyn classes with any PyTorch modules to build composite models. We offer additional tools to build custom neural differential equation and implicit models, including a functional API for numerical methods. There is much more in Torchdyn other than NeuralODE
and NeuralSDE
classes: tutorials, a functional API to a variety of GPU-compatible numerical methods and benchmarks.
Contribute to the library with your benchmark, tasks and numerical deep learning utilities! No need to reinvent the wheel :)
Installation
Stable release:
pip install torchdyn
Alternatively, you can build a virtual dev environment for torchdyn
with poetry, following the steps outlined in Contributing
.
Documentation
Check our docs for more information.
Introduction
Interest in the blend of differential equations, deep learning and dynamical systems has been reignited by recent works [1,2, 3, 4]. Modern deep learning frameworks such as PyTorch, coupled with further improvements in computational resources have allowed the continuous version of neural networks, with proposals dating back to the 80s [5], to finally come to life and provide a novel perspective on classical machine learning problems.
We explore how differentiable programming can unlock the effectiveness of deep learning to accelerate progress across scientific domains, including control, fluid dynamics and in general prediction of complex dynamical systems. Conversely, we focus on models powered by numerical methods and signal processing to advance the state of AI in classical domains such as vision of natural language.
By providing a centralized, easy-to-access collection of model templates, tutorial and application notebooks, we hope to speed-up research in this area and ultimately establish neural differential equations and implicit models as an effective tool for control, system identification and general machine learning tasks.
Dependencies
torchdyn
leverages modern PyTorch best practices and handles training with pytorch-lightning
[6]. We build Graph Neural ODEs utilizing the Graph Neural Networks (GNNs) API of dgl
[7]. For a complete list of references, check pyproject.toml
. We offer a complete suite of ODE solvers and sensitivity methods, extending the functionality offered by torchdiffeq
[1]. We have light dependencies on torchsde
[7] and torchcde
[8].
Applications and tutorials
torchdyn
contains a variety of self-contained quickstart examples / tutorials built for practitioners and researchers. Refer to the tutorial readme
Contribute
torchdyn
is designed to be a community effort: we welcome all contributions of tutorials, model variants, numerical methods and applications related to continuous and implicit deep learning. We do not have specific style requirements, though we subscribe to many of Jeremy Howard's ideas.
We use poetry
to manage requirements, virtual python environment creation, and packaging. To install poetry
, refer to the docs.
To set up your dev environment, run poetry install
. In example, poetry run pytest
will then run all torchdyn
tests inside your newly created env.
poetry
does not currently offer a way to select torch
wheels based on desired cuda
and OS
, and will install a version without GPU support. For CUDA torch
wheels,
run poetry run poe autoinstall-torch-cuda
, that will automatically install PyTorch based on your CUDA configuration.
If you wish to run jupyter
notebooks within your newly created poetry environments, use poetry run ipython kernel install --user --name=torchdyn
and switch the notebook kernel.
Choosing what to work on: There is always ongoing work on new features, tests and tutorials. If you wish to work on additional features not currently WIP, feel free to reach out on Slack or via email. We'll be glad to discuss details.
Cite us
If you find Torchdyn valuable for your research or applied projects:
@article{politorchdyn,
title={TorchDyn: Implicit Models and Neural Numerical Methods in PyTorch},
author={Poli, Michael and Massaroli, Stefano and Yamashita, Atsushi and Asama, Hajime and Park, Jinkyoo and Ermon, Stefano}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.