Skip to main content

torch-dd

PyPI

Double-double (emulated quad precision) arithmetic for PyTorch.

torch-dd represents each number as an unevaluated sum of two float64 values (hi + lo), giving roughly 106 significant bits — about twice the precision of float64 — while running on ordinary CPU and CUDA tensors. It is built on error-free transformations (two_sum, two_prod, Dekker split) and the QD-library algorithms, so results carry a proven relative-error bound instead of drifting.

Use it like a built-in numeric type through the DDTensor tensor subclass: operators, comparisons, torch functions, autograd and most of torch.nn.functional route through __torch_dispatch__ and run at DD precision. Build one with tensor(...) or the torch-style factories (zeros, ones, full, eye, arange, linspace, rand, randn, …); its .hi and .lo are the two float64 words, and .item() reads the value back as a rounded Python float.

import torch, torch_dd as dd

# float64 cannot even represent 1 + 2**-60, so the difference collapses to 0.
one = torch.ones(1, dtype=torch.float64)
tiny = torch.full((1,), 2.0**-60, dtype=torch.float64)
print(((one + tiny) - one).item())                       # 0.0   (wrong)

one_dd = dd.ones(1)
tiny_dd = dd.full((1,), 2.0**-60)
print(((one_dd + tiny_dd) - one_dd).item())              # 8.67e-19  (= 2**-60)

Why

float64 silently loses low-order bits in cancellation, long accumulations, and ill-conditioned formulas. The classic example is the small root of x² + 10⁸·x + 1 = 0 via the textbook formula: the subtraction amplifies the rounding error of sqrt by ~10¹⁶, so float64 keeps no correct digits while DD still keeps ~15.

import torch, torch_dd as dd

x = dd.full((1,), 1e8)
root = (-x + torch.sqrt(x * x - 4.0)) / 2.0                  # like builtin math
# relative error vs. the exact root:  float64 ~0.25   |   torch-dd ~4e-18

Install

pip install torch-dd
# or: uv add torch-dd

Requirements: Python ≥ 3.12, PyTorch ≥ 2.13. The core runs in pure PyTorch. On first use torch-dd JIT-compiles fused C++/CUDA kernels via torch.utils.cpp_extension for speed; if no compiler/CUDA toolkit is present it transparently falls back to the pure-PyTorch implementation, which produces bit-identical results.

Features

  • Operators & reductions: + - * / @, comparisons, sum, and matmul on a DDTensor, all with proven O(2**-106) error bounds.
  • Transcendentals: torch.exp, torch.log, torch.sin/cos/tan, torch.sinh/cosh/tanh, torch.rsqrt, torch.pow, torch.erf/erfc on a DDTensor evaluate in DD from mpmath-verified constants.
  • Autograd: reverse-mode differentiation runs at DD precision; gradients come back as DDTensors.
  • torch.nn.functional: forward and backward for the whole functional surface — activations, softmax, losses, normalizations, pooling, convolution, attention, interpolation, grid_sample, embeddings, ctc_loss, and more.
  • Fast matmul: @ uses an Ozaki-scheme product that turns one DD matmul into error-free lower-precision GEMMs — INT8 tensor cores on CUDA, blocked DGEMM on CPU — above a size threshold, with a dynamic-range guard and FP64 fallback.
import torch, torch_dd as dd
from torch_dd import DDTensor
import torch.nn.functional as F

w = dd.randn(4, 3, requires_grad=True)
x = dd.randn(2, 3)
F.mse_loss(F.linear(x, w), dd.zeros(2, 4)).backward()
assert isinstance(w.grad, DDTensor)      # a double-double gradient

Examples

examples/ trains an MNIST classifier entirely in double-double precision — weights, activations, gradients and optimizer state are all DDTensors — using a from-scratch DD AdamW optimizer:

python examples/train_mnist.py            # quick subset, a few seconds on GPU
python examples/train_mnist.py --full     # all 60k images (~97% test accuracy)

See examples/README.md for details.

Accuracy

Every operation stays within the DD contract: element-wise ops are correct to a few units of 2**-106 relative, and reductions/mm to O(k · 2**-105 · Σ|terms|). IEEE inf/nan propagate as in float64. The representable range matches float64 (the lo word degrades below ~2**-970, the documented double-double limit).

Development

uv run pytest             # the full test suite (CPU + CUDA)
uv run black src tests examples
uv run mypy               # file set and config live in pyproject.toml

References verify DD results against 256-bit mpmath (and exact Fraction arithmetic for the error-free transformations), well beyond float64 resolution.

Layout

src/torch_dd/
  eft.py        error-free transformations (two_sum, two_prod, split)
  core.py       DD type and arithmetic (add/mul/div/sqrt/sum/mm, ...)
  mathfn.py     transcendentals (exp/log/trig/hyperbolic/erf/pow)
  dd_const.py   exact hex-pair constants (mpmath-generated)
  tensor.py     DDTensor subclass + __torch_dispatch__ handlers
  factory.py    tensor-creation factories (zeros/ones/arange/linspace/...)
  ozaki.py      Ozaki-scheme mm engines (CUDA INT8 / CPU DGEMM)
  native.py     JIT loader for the fused C++/CUDA kernels
  csrc/         C++/CUDA kernel sources
examples/       DD AdamW + MNIST training
tests/          pytest suite (mpmath / Fraction references)

Metadata

Release files for torch-dd 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for torch-dd 0.1.1
File Size Uploaded Provenance Actions
torch_dd-0.1.1.tar.gz 955.9 kB With provenance Details for torch_dd-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for torch-dd 0.1.1
File Interpreter ABI Platform Actions
torch_dd-0.1.1-py3-none-any.whl Python 3 none any Details for torch_dd-0.1.1-py3-none-any.whl

Total release size: 1.0 MB

Release files / torch_dd-0.1.1.tar.gz

Download URL torch_dd-0.1.1.tar.gz
Size 955.9 kB
Tags Source
SHA-256 checksum
How to use checksums
aa77b69f3a7cf348b3bf9880bb6e1d9cd011ec6d6a4eb496c3d3ca3e6eb711ac
BLAKE2b-256 checksum
How to use checksums
e70917f6cbc4b3a77f55e260c48677ef443ab441fd841321ea7f56738fddb06c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 21, 2026.

Transparency log

Release files / torch_dd-0.1.1-py3-none-any.whl

Download URL torch_dd-0.1.1-py3-none-any.whl
Size 74.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
804e3b7ba4e53ff2130e71cf8b5502af42deaad3b00bf2643170065cbbbb52b0
BLAKE2b-256 checksum
How to use checksums
0c1c6239fbe1c9d1fb646cca61f95eadb383df5f36d92eb0766627a0f4538945
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 21, 2026.

Transparency log

Security for torch-dd 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for torch_dd-0.1.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jul 21, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page