specular-differentiation 0.11.9

Specular differentiation in normed vector spaces and its applications

Specular Differentiation

The Python package specular implements specular differentiation which generalizes classical differentiation. This implementation strictly follows the definitions, notations, and results in [1] and [2].

A specular derivative (the red line) can be understood as the average of the inclination angles of the right and left derivatives. In contrast, a symmetric derivative (the purple line) is the average of the right and left derivatives. Their difference is illustrated as in the following figure.

Also, specular includes the following applications:

• Ordinary differential equation

  • Directory: examples/ode/
  • References: [1], [3], [4]

• Optimization

  • Directory: examples/optimization/
  • References: [2], [5]

Installation

Requirements

specular-differentiation requires:

• Python >= 3.11
• ipython >= 8.12.3
• matplotlib >= 3.10.8
• numpy >= 2.4.0
• pandas >= 2.3.3
• tqdm >= 4.67.1

User installation

The package is available on PyPI:

pip install specular-differentiation

Quick start

The following simple example calculates the specular derivative of the ReLU function $f(x) = max(0, x)$ at the origin.

>>> import specular
>>> 
>>> ReLU = lambda x: max(x, 0)
>>> specular.derivative(ReLU, x=0)
0.41421356237309515

Tutorial

Detailed usage examples can be found in documentation.

Applications

Specular differentiation is defined in normed vector spaces, allowing for applications in higher-dimensional Euclidean spaces. Two applications are provided in this repository.

Ordinary differential equation

In [1], seven schemes are proposed for solving ODEs numerically:

• the specular Euler scheme of Type 1~6
• the specular trigonometric scheme

The following example shows that the specular Euler schemes of Type 5 and 6 yield more accurate numerical solutions than classical schemes: the explicit and implicit Euler schemes and the Crank-Nicolson scheme.

Optimization

In [2], three methods are proposed for optimizing nonsmooth convex objective functions:

• the specular gradient (SPEG) method
• the stochastic specular gradient (S-SPEG) method
• the hybrid specular gradient (H-SPEG) method

The following example compares the three proposed methods with the classical methods: gradient descent (GD), Adaptive Moment Estimation (Adam), and Broyden-Fletcher-Goldfarb-Shanno (BFGS).

LaTeX symbol

To use the specular differentiation symbol in your LaTeX document, add the following code to your preamble (before \begin{document}):

% Required packages
\usepackage{graphicx}
\usepackage{bm}

% Definition of Specular Differentiation symbol
\newcommand\sd[1][.5]{\mathbin{\vcenter{\hbox{\scalebox{#1}{\,$\bm{\wedge}$}}}}}

Usage examples

Use the symbol in your document (after \begin{document}):

% A specular derivative in the one-dimensional Euclidean space
$f^{\sd}(x)$

% A specular directional derivative in normed vector spaces
$\partial^{\sd}_v f(x)$

References

[1] K. Jung. Nonlinear numerical schemes using specular differentiation for initial value problems of first-order ordinary differential equations. arXiv preprint arXiv:??, 2025.

[2] K. Jung. Specular differentiation in normed vector spaces and its applications to nonsmooth convex optimization. arXiv preprint arXiv:??, 2025.

[3] K. Jung and J. Oh. The specular derivative. arXiv preprint arXiv:2210.06062, 2022.

[4] K. Jung and J. Oh. The wave equation with specular derivatives. arXiv preprint arXiv:2210.06933, 2022.

[5] K. Jung and J. Oh. Nonsmooth convex optimization using the specular gradient method with root-linear convergence. arXiv preprint arXiv:2210.06933, 2024.