PyDiffGame 2.0.3

Nash-equilibrium solutions to linear-quadratic differential games, via a reduction of the Game Hamilton-Jacobi-Bellman equations to coupled algebraic and differential Riccati equations for multi-objective dynamical control systems.

Nash-equilibrium control for multi-objective dynamical systems, built on coupled Riccati equations.

---

• What is this?
• Why differential games?
• Installation
• Quick start
• Input parameters
• Tutorial: masses on springs
• More examples
• Testing and development
• Citing
• Acknowledgments
• Star history

What is this?

PyDiffGame is a Python implementation of a Nash-equilibrium solution to differential games. It reduces the Game Hamilton–Jacobi–Bellman (GHJB) equations to a set of coupled Game Algebraic and Differential Riccati equations, and solves them to synthesize feedback controllers for multi-objective dynamical control systems.

In one sentence: where a classical Linear-Quadratic Regulator (LQR) folds every control task into one quadratic cost and solves one Riccati equation, PyDiffGame keeps each task as a separate player, and solves the coupled Riccati system whose fixed point is their Nash equilibrium. A plain LQR is simply the one-player special case.

The method follows the formulation in:

• The thesis “Differential Games for Compositional Handling of Competing Control Tasks” (ResearchGate)
• The conference paper “Composition of Dynamic Control Objectives Based on Differential Games” (IEEE · ResearchGate)

The package requires Python ≥ 3.11 and is tested on CPython 3.11, 3.12, 3.13 and 3.14.

Why differential games?

Classical LQR	PyDiffGame
A single, hand-blended cost $\int x^\top Q x + u^\top R u$	One objective per task / player, each with its own $Q_i, R_i$
One Algebraic Riccati Equation	A coupled system of Riccati equations solved to its Nash fixed point
Re-tune the whole weight matrix to add a task	Compose tasks by adding a player — the others keep their own cost
Continuous time only, by convention	Continuous and discrete time, finite or infinite horizon

PyDiffGame ships both regulation (drive the state to the origin) and signal tracking (drive the state to a target $x_T$), a one-call LQR-vs-game comparison harness, cost accounting on a common yardstick, and ready-made plotting.

Installation

PyDiffGame is published on PyPI and is managed with uv. Add it to your project:

uv add PyDiffGame

To run the bundled examples (which additionally need python-control):

uv add "PyDiffGame[examples]"

Prefer pip? It works as a fallback.

pip install PyDiffGame
pip install "PyDiffGame[examples]"   # with the examples extra

To work on the package itself, clone it and sync the locked development environment with uv:

git clone https://github.com/krichelj/PyDiffGame.git
cd PyDiffGame
uv sync --extra dev          # creates .venv with the exact locked dependencies
uv run pre-commit install    # enable the formatting / lint / type-check hooks

Then run anything through uv run (uv run pytest, uv run python -m PyDiffGame.examples.MassesWithSpringsComparison, …). Pip users can instead pip install -e ".[dev]", though uv is recommended for the exact locked environment.

Quick start

A plain Linear-Quadratic Regulator is just a one-objective game. The continuous solver matches scipy.linalg.solve_continuous_are exactly:

import numpy as np
from PyDiffGame import ContinuousLQR

A = np.array([[0.0, 1.0],
              [0.0, 0.0]])
B = np.array([[0.0],
              [1.0]])

lqr = ContinuousLQR(A=A, B=B, Q=np.eye(2), R=1.0).solve()

print(lqr.K[0])                     # optimal feedback gain
print(lqr.is_closed_loop_stable())  # True

For a multi-player differential game, give one Objective per player. Each player owns a slice of the physical input through a decomposition matrix M:

import numpy as np
from PyDiffGame import ContinuousPyDiffGame, GameObjective

A = np.array([[0.0, 1.0, 0.0, 0.0],
              [0.0, 0.0, 0.0, 0.0],
              [0.0, 0.0, 0.0, 1.0],
              [0.0, 0.0, 0.0, 0.0]])
B = np.eye(4)[:, [1, 3]]

objectives = [
    GameObjective(Q=np.diag([1.0, 0.1, 0.0, 0.0]), R=1.0, M=np.array([[1.0, 0.0]])),
    GameObjective(Q=np.diag([0.0, 0.0, 1.0, 0.1]), R=1.0, M=np.array([[0.0, 1.0]])),
]

game = ContinuousPyDiffGame(A=A, objectives=objectives, B=B).solve()

# Each converged P_i drives its coupled algebraic Riccati residual to ~0:
print(max(np.max(np.abs(r)) for r in game.algebraic_riccati_residuals()))  # ~1e-14
print(game.is_closed_loop_stable())                                        # True

Input parameters

A game is described by a system matrix A, a set of Objective objects (one per player), and an input description (B together with each objective's decomposition matrix M, or per-player matrices Bs). Construct one of the concrete solvers — ContinuousPyDiffGame or DiscretePyDiffGame — with the following parameters:

Parameter	Type	Meaning
A	np.ndarray $(n, n)$	System dynamics matrix
objectives	Sequence[Objective] of length $N$	One objective per player; each carries $Q_i$, $R_{ii}$ and optionally $M_i$
B	np.ndarray $(n, m)$, optional	Full input matrix, used with each objective's decomposition matrix $M_i$
Bs	Sequence[np.ndarray], optional	Per-player input matrices $B_i$ of shape $(n, m_i)$ — an alternative to B + M
x_0	np.ndarray $(n,)$, optional	Initial state vector
x_T	np.ndarray $(n,)$, optional	Final (target) state for signal tracking
T_f	positive float, optional	Finite-horizon length; omit (or None) for the infinite-horizon problem
P_f	Sequence[np.ndarray], optional	Terminal Riccati condition (default: uncoupled algebraic Riccati solutions)
L	positive int, default 1000	Number of time samples
eta	positive int, default 5	Number of trailing matrix norms inspected for convergence
epsilon_x, epsilon_P	float in $(0, 1)$, optional	Convergence tolerances for the state and the Riccati matrices
state_variables_names	Sequence[str] of length $n$, optional	LaTeX names (without $) for state variables, used in plots
show_legend	bool, default True	Whether plots include a legend
debug	bool, default False	Emit verbose diagnostics while solving

An Objective takes:

• Q — np.ndarray $(n, n)$, symmetric positive semi-definite state weight
• R — np.ndarray $(m_i, m_i)$ (or a scalar), symmetric positive definite input weight
• M — np.ndarray $(m_i, m)$, optional decomposition matrix (None for a plain LQR objective)

The helpers LQRObjective(Q, R) and GameObjective(Q, R, M) are thin constructors for the two common cases.

Tutorial: masses on springs

To show the package in action we compare a differential game against an LQR on a chain of masses connected by springs — a textbook coupled, oscillatory system:

The physical input space is decomposed along the modal directions of $M^{-1}K$, so each vibration mode becomes one player of a game. The full example lives in MassesWithSpringsComparison.py; here is its essence:

import numpy as np
from PyDiffGame import GameObjective, LQRObjective, PyDiffGameLQRComparison

N, m, k, r = 2, 50.0, 10.0, 1.0
q = [500.0, 2000.0]                       # per-mode state weights

I_N, Z_N = np.eye(N), np.zeros((N, N))
mass_inv_stiffness = (1.0 / m) * k * (2 * I_N - np.eye(N, k=1) - np.eye(N, k=-1))

# Modal decomposition (orthonormal, so the modal transform is orthogonal):
_, eigenvectors = np.linalg.eigh(mass_inv_stiffness)
Ms = [eigenvectors[:, i].reshape(1, N) for i in range(N)]
modal_to_state = np.kron(np.eye(2), np.concatenate(Ms, axis=0))

A = np.block([[Z_N, I_N], [-mass_inv_stiffness, Z_N]])
B = np.block([[Z_N], [(1.0 / m) * I_N]])

game_objectives = []
for i, (q_i, M_i) in enumerate(zip(q, Ms)):
    modal_weight = np.diag([0.0] * i + [q_i] + [0.0] * (N - 1) + [q_i] + [0.0] * (N - i - 1))
    game_objectives.append(GameObjective(Q=modal_to_state.T @ modal_weight @ modal_to_state, R=r, M=M_i))

# The LQR baseline optimizes exactly the aggregate of the game's objectives
# (sum of the per-mode Q_i, and the matching physical input weight r * I), so
# it is the genuine monolithic optimum to compare the decomposed game against:
modal_weight_total = np.diag(q + q)
lqr_objective = [LQRObjective(Q=modal_to_state.T @ modal_weight_total @ modal_to_state, R=r * I_N)]

x_0 = np.array([10.0, 20.0, 0.0, 0.0])
comparison = PyDiffGameLQRComparison(
    A=A, B=B,
    games_objectives=[lqr_objective, game_objectives],
    x_0=x_0, x_T=x_0 * 10.0, T_f=25.0, L=300,
)

comparison.run(plot_state_spaces=True)
lqr_cost, game_cost = comparison.costs()
print(f"LQR cost = {lqr_cost:.4g},  game cost = {game_cost:.4g}")

Run the bundled example end-to-end:

uv run python -m PyDiffGame.examples.MassesWithSpringsComparison

The monolithic LQR is, by construction, the optimal controller for the aggregate of the game's objectives. The fully decomposed modal game — where each vibration mode is solved as an independent player, coupled only through the shared dynamics — reproduces that monolithic optimum to numerical precision: the two state trajectories coincide (they differ by ~10⁻⁷) and the costs are equal:

For this modally-decoupled system the decomposition is lossless — and it buys compositionality: you can add, drop or re-weight a control task by editing a single player, without re-tuning one monolithic cost matrix.

The figures above are regenerated from the live solver by tools/generate_readme_figures.py (uv run python tools/generate_readme_figures.py), so they always match the current code.

More examples

The src/PyDiffGame/examples directory contains further worked comparisons:

Example	System
MassesWithSpringsComparison.py	Chain of masses coupled by springs (the tutorial above)
InvertedPendulumComparison.py	Inverted pendulum on a cart
PVTOL.py · PVTOLComparison.py	Planar vertical take-off & landing aircraft
QuadRotorControl.py	Quadrotor attitude / position control

Testing and development

The package ships with a pytest suite that validates the solvers against analytical ground truth — LQR solutions are checked against scipy's algebraic Riccati solvers, and the games against their coupled-Riccati residuals and closed-loop stability. All tooling runs through uv:

uv sync --extra dev
uv run pytest                                # run the test suite
uv run ruff format src/PyDiffGame tests      # auto-format (black-compatible)
uv run ruff check  src/PyDiffGame tests      # lint
uv run mypy        src/PyDiffGame            # type-check

Continuous integration runs the formatter check, linter, type checker and full suite on Python 3.11–3.14. See CONTRIBUTING.md for details.

Citing

If you use this work, please cite our paper:

@inproceedings{pydiffgame_paper,
    author={Kricheli, Joshua Shay and Sadon, Aviran and Arogeti, Shai and Regev, Shimon and Weiss, Gera},
    booktitle={29th Mediterranean Conference on Control and Automation (MED 2021)},
    title={{Composition of Dynamic Control Objectives Based on Differential Games}},
    year={2021},
    pages={298-304},
    doi={10.1109/MED51440.2021.9480269}}

Further details can be found in the citation document.

Acknowledgments

This research was supported in part by the Leona M. and Harry B. Helmsley Charitable Trust through the ‘Agricultural, Biological and Cognitive Robotics Initiative’ (‘ABC’) and by the Marcus Endowment Fund, both at Ben-Gurion University of the Negev, Israel. It was also supported by The ‘Israeli Smart Transportation Research Center’ (‘ISTRC’) by The Technion and Bar-Ilan Universities, Israel.

   

Star history