---

torchnative replaces PyTorch's compiled core — torch._C — with a native extension, so the genuine torch and transformers packages run on a phone the way they run on a workstation.

Models are not ported, converted, or re-expressed. They are imported.

from transformers import AutoModelForCausalLM     # the real one
model = AutoModelForCausalLM.from_pretrained("...")
model.generate(...)                                # on the device

[!WARNING] Pre-alpha. The operator layer matches upstream PyTorch numerically, 19 of 20 tested architectures reach zero missing operators, real checkpoints load, transformers imports and generates, and an Android device runs the built artefact — but there is no accelerator backend, torch.compile does not work, and of the nine build targets two have never been executed anywhere (Windows arm64, iOS device) and one refuses to build at all (Android x86_64). See Status, Platform support and docs/platform/WHEELMATRIX.md before depending on this.

---

Why not a reimplementation

Every other route to on-device inference re-expresses the model somewhere else.

	approach	cost
llama.cpp	architectures rewritten in C++	each new architecture is a porting task
ExecuTorch · CoreML	ahead-of-time compiled graph	export step, and what runs is not what you wrote
MLC	lowered to its own runtime	same
torchnative	the real Python package	the substrate is hard; architectures are free

The reason nobody runs the real thing is that torch._C cannot be built for mobile. PyTorch's own build sets INTERN_BUILD_MOBILE for any Android or iOS toolchain, and that path forces BUILD_PYTHON off — so the mobile build is structurally incapable of producing the Python extension module the Python package needs.

torchnative supplies that module instead. Everything above it is upstream source, unmodified.

---

What it does

1 · LLM inference

Run transformers models directly. No conversion step, no per-architecture port — if transformers supports it and the operators are covered, it runs.

Tokens arrive one at a time, the way an app wants them:

import torch
from threading import Thread
from transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer

name  = "HuggingFaceTB/SmolLM2-135M"
model = AutoModelForCausalLM.from_pretrained(name, dtype=torch.float32)
tok   = AutoTokenizer.from_pretrained(name)

streamer = TextIteratorStreamer(tok, skip_prompt=True, skip_special_tokens=True)
inputs   = tok("On-device inference is", return_tensors="pt")

Thread(target=model.generate, kwargs=dict(**inputs, max_new_tokens=32, streamer=streamer)).start()

for piece in streamer:          # yields as the model decodes
    print(piece, end="", flush=True)

Today: this is the output of that exact script, not a sketch of it —

 a very powerful technique for learning from data. It is a powerful technique for
 learning from data because it is a very fast and efficient way to learn from data.

first token in 34 ms, then 47 tokens/second on an M-series desktop. generate runs on a background thread and the main thread consumes the iterator, so the shim is holding up under two threads and the GIL, not only under a single-threaded loop.

The weights come from the Hub through from_pretrained — 273 tensors, bit-identical to upstream — and in float32 the tokens are the same ones upstream emits.

Loaded in the checkpoint's native bfloat16, which is what you get if you pass no dtype, the tokens diverge. That is not a defect to fix: upstream disagrees with itself on one prompt in three under a mathematically equivalent change of accumulation order, so bitwise agreement in bf16 is not a bar any independent implementation can clear. See Status.

2 · Federated learning

Devices train locally and share updates, not data. Federated averaging is collective communication, so this is built on torch.distributed rather than beside it — broadcast the model, gather the updates, weighted all-reduce.

from torchnative.nn import federated

engine = federated.Engine(model, method=adapt.Tent(), aggregator=federated.FedAvg())
report = engine.participate(batches, weight=n_local_samples)   # local epochs, then a delta

Today: one round runs, between two operating-system processes that share no memory. Each rank adapts locally with adapt.wrap(model, method=Tent()), contributes the delta that produces, and comes back holding the group's weighted average — and the acceptance check is not that the distributed path returned something. It is that

aggregate_across_the_two_ranks  ==  (3·d0 + 7·d1) / 10

element for element, with the right-hand side computed centrally in a third process on upstream torch from the two deltas the ranks dumped. torch.equal, not a tolerance: every operation on both sides is a correctly-rounded IEEE float32 multiply, add or divide, so one ulp apart would be a real disagreement. Both ranks land on the same bits, and after the round they hold the same model (docs/distributed/FEDERATED.md).

The trap was that FedAvg at world_size = 1 is the identity function — it returns the delta it was handed, so a test at that size passes whether the weights are honoured, ignored, or never read. So a world of one is refused at all four doors rather than served, two threads are not enough either, and the controls are part of the claim: the weighted average differs from the unweighted one by 0.041 and 0.165 on the two covered parameters, so an aggregator that dropped its weights fails. Five injected defects were counted, and the two that removed an agreement check made the mismatch complete silently — different parameter sets summed into a number with no exception.

The three things it was waiting on are all closed: torch.save works and upstream reads what it writes bit-for-bit across eight dtypes (docs/models/SAVE.md); world_size = 2 works over a real socket, through the ordinary init_process_group(backend="local", init_method="tcp://…") and not a private door (docs/distributed/TRANSPORT.md); and Delta.publish, the seam, now sends.

What it does not cover, by name: more than one round, participant selection, dropout handling, secure aggregation, and any aggregator that is not FedAvg — each refusing with what it would take rather than approximating. A rank that does not arrive makes the round raise; it never produces a partial average. And a delta above ~2 MB on the wire refuses, because the transport under it sends before it receives and deadlocks — a limit of ProcessGroupLocal, named there rather than worked around here.

3 · Test-time adaptation & training

A model that ships to a device meets data the training set never had. TTA, TTT and the wider test-time learning family let it adapt in place — and every method reduces to the same thing: a weight delta over base weights, differing only in lifetime and destination.

from torchnative import adapt

model = adapt.wrap(model, method=adapt.Tent(), lr=1e-3)
model.online()                                   # adapt as it serves
model.revert()                                   # the base weights are back, byte for byte

Today: Tent runs on a real checkpoint. On SmolLM2-135M, ten steps of entropy minimisation over the 61 normalisation weights take prediction entropy on unlabelled text from 4.1604 to 2.9828, and a held-out sentence the loop never adapted on falls 3.7237 to 2.9439 — so the model adapted rather than memorising one batch. The adapted weights agree with upstream's own autograd running the same step to a median relative 1.5e-06 with 100% sign agreement over 35,136 numbers. The controls are part of the claim: the same code with the objective's sign flipped sends entropy up to 7.4062, lr=0 holds it identical to the last printed digit, and an objective on a detached tensor is refused by name rather than running vacuously (docs/models/ADAPT.md).

The delta abstraction of docs/design/DESIGN.md §3 is what carries it: torchnative.delta.Delta owns the base, the offset, and the three lifetime questions, so Tent holds no state and is 40 lines. A revert restores the base bit-identically — all 272 parameters, not only the 61 covered — and the base copy costs 137 KiB against the model's 513 MiB. Lifetime is driven by system events rather than by the domain boundaries a benchmark hands you, and the two lifetimes that do not exist yet (surviving a restart, leaving the device) refuse with the check that would prove them stale.

What it does not cover: any nn.LayerNorm model. aten.native_layer_norm.default has no derivative rule, so every RMSNorm architecture adapts and gpt2/bert are refused before the backward, by name.

---

How it works

your code · transformers · torch/*.py        upstream Python, unmodified
──────────────────────────────────────────
torch._C                                     ← replaced
  ├── _aten_dispatch                         the single door every operator passes
  ├── Python spellings                       torch.mm, x.softmax(), F.linear, ...
  └── kernels                                Rust, backed by candle
──────────────────────────────────────────
CPU today · Metal, Vulkan, NPU planned

One door. Every operator reaches its kernel through _aten_dispatch, and nothing bypasses it. That makes the surface measurable — an unimplemented operator names itself rather than failing downstream — and it gives graph capture, which NPU backends will need, exactly one place to attach.

Demand-driven. Nothing is implemented because it might be needed. The shim refuses by name, the refusal names the next thing to build, and that list comes from running real models.

Stable ABI. Built against CPython's limited API (abi3-py313), so one binary per platform loads on 3.13, 3.14 and later without a rebuild.

---

Status

Working
ATen operators	302, each compared against upstream
Golden comparison cases	11,420 / 11,420 — values, shapes, dtypes, positional and keyword, through the door and through the member. The golden_cases_failed marker below is the one that matters: for a while the only two markers here were ge floors on passed and on total, and a pair of floors cannot see passed < total. One case failed for three commits with the gate green
Smoke tests	929 across 30 files — 480 of them in test_shim.py, which is what the marker below counts, and the rest in the files split off it, one per round. The split exists because reconstructing a single conflict hunk in one large file had twice silently dropped tests
from_pretrained	works for models whose init computes on the meta device — the Llama-3.2 rope_scaling path needed 30-odd meta kernels that were absent (META.md)
Signature and schema tables	5,024 of 5,037 entries checked against upstream
Architectures — operator coverage	26 of 26 reach zero missing operators in the traced sweep
Architectures — agreeing with upstream	26 of 26, matching upstream. Agreement is module-by-module through forward hooks, because two of the toy outputs are degenerate enough that their argmax is a tie — reported as a tie rather than as a match (KERNELS26.md)
Architectures — swept, all of them	297 of 297 forward (100%) — docs/architectures/ARCH300.md recorded 290 (98%), and the re-measurement below adds four; up from 270/297 (91%) in ARCH200 and 215/297 (72%) in ARCH100. The denominator is not 528. 528 is every model type AutoModel can build; of those, 231 fail on upstream torch too under the same shrunk random-weight config, so they are not this project's gap and are excluded — 294 of 528 would be a different and wrong claim. ARCH300's remaining 7 were blocked on argument forms and kernels, three of them sharing one argument-form gap (a tensor/tuple passed where the shim's table has no matching row), which is a first-wall count: closing one wall can reveal another. And a forward is not a match — this row measures reachability only; the row below measures agreement (ARCH300.md, prior rounds ARCH200.md, ARCH100.md). Re-measured at release time on the current head: the seven were each re-run individually and four of them now forward (univnet, nystromformer, vilt, sam3_lite_text_text_model), taking that step to 294 of 297. The other 290 were not re-swept, so 294 rests on ARCH300's 290 plus four individual runs rather than on a fresh full sweep. The three then still blocked have since been closed and all 297 forward — fastspeech2_conformer needed repeat_interleave with a tensor repeats, and led/longformer needed Tensor.where and then an as_strided size element arriving as a 0-dim tensor, ARCH300's own first-wall caveat firing twice more. The same qualification carries: 297 rests on ARCH300's 290 plus seven individual runs, not on a fresh full sweep (REPEAT.md)
Architectures — numerically agreeing with upstream	284 of 285 judgeable architectures agree (99.6%). Every architecture that forwards was run on both sides with the same weights and the same inputs — the state_dict travels as bytes, and transferred with no missing and no unexpected key for 290 of 290 — and compared element-wise. Until this measurement, every coverage number this project published measured reachability: “it imports and runs” and “it computes upstream’s numbers” are different claims, and only the second supports the word drop-in. The tolerance is derived, not chosen: each architecture was additionally run upstream in float64, and the threshold is the p90 of upstream’s own float32-vs-float64 error over the 263 architectures with a working oracle, floored at 8 ulp — 1.19e-06. Anything tighter would have to call upstream wrong on a tenth of the same set. Nine architectures are closer to the float64 answer than upstream is. And the result is negative about operators: replaying every leaf module on upstream’s own recorded input, so nothing accumulates, the worst single-operator error anywhere in 775 replayed leaf modules is 8.4 ulp. The large end-to-end numbers are float32 accumulation over depth, not defects. The caveats, kept rather than absorbed: 5 architectures could not be judged and are excluded from the denominator rather than counted as passes — three whose output underflowed (both sides agree on noise) and two where upstream does not reproduce itself (vit_mae re-draws its patch mask, vits samples a duration). 22 MoE models have no float64 oracle at all, because upstream refuses Double at its grouped matmul, so only the fixed tolerance applies to them. And chinese_clip is left flagged as the one divergence even though the round found the flag spurious — its absolute difference is twelve ulp and no operator in it exceeds 2.5 ulp; it crossed the rule because upstream’s own oracle error on that output is unusually small. Tuning a rule until a flag disappears is not a result (AGREE.md)
Checkpoints	torch.load and safetensors, round-tripped against upstream
Build targets	macOS · Android · iOS · Linux · Windows — eight of nine targets build a wheel (the ninth, Android x86_64, refuses by name — WHEELMATRIX.md §3.3) — build.py --target wasm32-emscripten now produces the WASM one, so the sentence that it could not is no longer true. What has not happened is anything importing that build.py-produced wheel under Pyodide: the computing claim for WASM still rests on the earlier hand-built wheel (table)
Training mode	26 of 26 forward in .train() as well as .eval(), agreeing with upstream draw for draw — bernoulli_ draws in float64 for every dtype, so a seeded dropout is comparable. Test-time adaptation runs on real checkpoints — adapt.wrap(model, method=adapt.Tent()) drops GPT-2's prediction entropy 39% and transfers to held-out text — in .train() as well as .eval(), with dropout active. A training step moves all 272 SmolLM2 parameters the way upstream moves them — gradients compared element-wise over all 134,515,008 values, sign agreement 99.9987%. loss.backward() now works, through upstream's own path (torch/_tensor.py → _engine_run_backward → _ImperativeEngine.run_backward) with no shim-specific call: a six-step SGD loop over an nn.Sequential, driven by the real torch.optim.SGD, matches upstream to 2.98e-08 — one float32 ulp — across the loss trajectory, the gradients and the final parameters. What it is and is not, re-measured at release time rather than restated: a transformer does now train through loss.backward() — a BERT encoder built from a shrunk config runs three zero_grad/backward/step iterations and its loss trajectory matches upstream's to float32 (5.12 → 0.0 → −5.12 on both sides), with a gradient on 21 of 23 parameters and the two without one being the unused pooler, which upstream also leaves ungradiented. That was measured with dropout disabled: with dropout on, the two sides diverge after the first step because the RNG streams differ, which is a sampler difference and not a gradient defect. Convolution backward landed after that sentence was drafted: a small CNN with strided, depthwise and pointwise convolutions, three batch-norms in training mode and a linear head trains end to end through five SGD steps, agreeing with upstream to 2.98e-08 (TRAIN2.md). Still absent: create_graph=True/double-backward, multiple root tensors, GradientEdge inputs, torch.autograd.Function, hooks and retain_grad on non-leaves all refuse by name. Mutation through a view is refused rather than differentiated, which is deliberately less than upstream (BACKWARD9.md, BACKWARD7.md) Unlike torch.compile, autograd is reachable under abi3 — torch/csrc/autograd defines Py_BUILD_CORE in 0 of 129 files — and a SmolLM2 backward needs 24 ops of which 16 exist and one is a real missing kernel (AUTOGRAD.md)
Test-time adaptation	Tent runs on SmolLM2-135M. Ten steps of entropy minimisation over the 61 normalisation weights: entropy 4.1604 → 2.9828 on unlabelled text, 3.7237 → 2.9439 on a held-out sentence never adapted on, adapted weights within a median relative 1.5e-06 of upstream's own autograd at 100% sign agreement. Reverting restores the base bit-identically across all 272 parameters, for a 137 KiB base copy against 513 MiB of model. The wrong sign sends entropy up, lr=0 holds it to the last digit, and a detached objective is refused by name — because a loop that silently does nothing passes every test that only checks it completed. nn.LayerNorm models are refused: aten.native_layer_norm.default has no derivative rule (ADAPT.md)
Accelerators	Metal and Vulkan compute on this Mac's real GPU. mps is candle's Metal backend; vulkan is a fourth arm of tensor::Repr outside candle, with a real VkBuffer round-trip. Both are gated so a silent CPU fallback cannot happen: Vulkan teaches eighteen ops by name (eleven of them compute kernels) and refuses the rest naming themselves, and a whole nn.Sequential(Linear, ReLU, Linear) forwards on it with zero host readbacks, and on mps every op whose kernel would read the tensor back to the host is refused by name — enumerable at runtime through _C._shim_mps_host_readback_ops(). A transformer does not run on mps yet: aten._softmax.default is in that refused set, and every attention block passes through it (MPS.md, VULKAN3.md)
NPU	The capture layer exists and a graph lowers through it. A CoreML .mlpackage is compiled by macOS and executed through MLModel.predict, agreeing with the replayed trace to 2–3e-08 at float32 — and float32 had to be forced, because coremltools defaults to float16, which is four orders of magnitude looser. The NNAPI blob is structurally validated only: there is no NNAPI runtime on a Mac and nothing here has run on an NPU (NPU.md)
torch.distributed	ProcessGroupLocal at world_size >= 3, over real loopback TCP sockets in a star with the hub at rank 0, folding contributions in ascending rank order so the answer does not depend on arrival order. Proper-subset cohorts, a survivor set after a dropout, and on_missing='average_arrived' all run. Everything else — other reduce ops, broadcast/allgather/send/recv, secure aggregation, differential privacy — refuses by name (FEDERATED4.md)
Devices run	Android arm64 — import torch, 119 ops, nn forward. WASM runs under Pyodide — a hand-built wheel installs, imports and computes, on CPython 3.14
Speed vs upstream	desktop CPU, SmolLM2-135M prefill: 0.97x at 6 tokens, 1.13x at 128, 1.52x at 512, 2.03x at 1024 in float32 — the gap grows with sequence length and what is left is attention (SEQLEN.md). In bfloat16 it is 2.3x faster than upstream (DTYPE_PERF.md). Decode is the other half and it was never measured until now: generate() with a KV cache — the default, and what the example above runs — is 0.95x, 46.6 tok/s against upstream's 44.4 on SmolLM2-135M float32, with character-identical output. The long-sequence gap is attention, and not because we materialise the score matrix: two independent blocked kernels were built to stop materialising it and both were slower — upstream's own, reproduced exactly, by 20x (FLASH.md)

Twenty of the twenty-six checked for agreement: Llama · GPT-2 · Qwen2 · Mistral · Gemma · GPT-NeoX · OPT · MPT · StarCoder2 · StableLM · OLMo · Phi · Mixtral · BERT · BLOOM · Cohere · Falcon · Mamba · Persimmon · GPT-BigCode

The two rows measure different things, and conflating them is a mistake this README made. The coverage sweep traces a forward pass on upstream torch and asks whether every operator it dispatches is implemented here — so it cannot see anything that is not an operator: an unbound tensor member, a missing torch.<name> spelling, a dtype-promotion rule.

Closing the six took 11 new kernels, 12 spellings, 16 tensor members, 13 _C surface names and 3 rule changes — and none of the six stopped on only one wall. Each had one to five more behind it, of a different kind each time: Cohere needed three spellings and no kernel at all, BERT went surface then spelling then kernel. "One operator away" was never true of any of them (docs/architectures/ARCH20.md).

Measured against transformers 5.x, which is what a fresh pip install transformers resolves today. 4.x costs four more architectures and needs a disjoint set of operators from Mixtral (docs/models/COMPAT.md).

uniform_ and normal_ are bit-identical to upstream, and multinomial consumes the same generator stream — a seeded run reproduces exactly. randn, rand, their _like forms and torch.normal are composed from those, and agree with upstream value for value under a seed.

Not working yet

• torch.compile does not work, and the reason is structural rather than a missing piece. Dynamo's frame-evaluation hook needs CPython internals — all six C files under torch/csrc/dynamo define Py_BUILD_CORE, and set_eval_frame reaches _PyInterpreterState_SetEvalFrameFunc on a _PyInterpreterFrame — which cannot coexist with the limited API in one extension. torch.compile and abi3 are mutually exclusive, and abi3 is what lets one binary per platform serve 3.13 and every later CPython. Eager is the supported path, and graph capture through the single door — already bit-exact against eager — is the route being pursued instead (docs/graph/DYNAMO.md). That is now a recommendation to refuse it permanently, not a postponement: docs/graph/COMPILE.md says ship abi3 only, refuse torch.compile by name, and spend the effort on torch.export. Nothing here has ever implemented any part of either — torch.export is reachable under abi3, 18 symbols censused with none in a Py_BUILD_CORE file, but a census is not an implementation (docs/graph/COMPILE.md).
• The GPU is on, and it is not a backend you can run a model on. Metal and Vulkan compute, under gates that refuse rather than fall back — see the Status table. aten._softmax.default is refused on mps, so no transformer forwards there.
• 7 of 297 architectures do not forward, down from 27/16 in ARCH200 and 82/31 in ARCH100. See the Status table for what 297 means.
• The Android run is an emulator, not a phone. No number here describes real silicon.
• Apple is much faster than Android at f32 matmul, and that is the hardware. Accelerate reaches the AMX coprocessor; ARMv8.2-A NEON has no equivalent. Our Android throughput equals our own throughput on the same core under the same backend, at 88% of that core's NEON peak — so the kernels are not the gap. Upstream PyTorch has no Android wheel, so how we compare to it there is unmeasured. See docs/perf/PERF_ANDROID.md.
• Speed work transfers to Android, but not uniformly. Dispatch-bound wins arrive slightly larger on device than on the host; kernel- and bandwidth-bound ones arrive smaller — the attention copy is 3.6x there against 5.25x here. Measured by swapping one .so between published wheels, which land the optimisations one at a time (docs/perf/PERF_ANDROID.md §10).

Tracked with the measurements behind them in docs/design/DESIGN.md §11.1.

---

Platform support

Three axes, and they are not independent: a dtype only means something on a device, and a device only exists on a platform. Every ✅ has a run behind it.

Legend — ✅ measured working · ❌ measured refusing · ⚠️ built, never executed · 🔲 not built · — not applicable to that platform

Platforms

	macOS arm64	Android arm64	Android x86_64	iOS sim arm64	iOS device arm64	Linux x86_64	Linux aarch64	Windows x86_64	Windows arm64	WASM
in the target matrix	✅	✅	✅ listed, refuses	✅	✅	✅	✅	✅	✅	— deliberately
rust target installed	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
target CPython	✅	✅	❌ none exists, none downloadable	✅	✅	✅	✅ PBS 20260825	✅	✅ PBS 20260825	✅ Pyodide 3.14
candle builds	✅	✅	🔲	✅	✅	✅	✅	✅	✅	✅
candle computes	✅	✅	🔲	✅	⚠️	✅ CI	✅ here	✅ CI	⚠️	✅ under Node
CUDA extension builds	—	—	—	—	—	🔲 CI job written, never run	🔲 not targeted	🔲	—	—
CUDA computes	—	—	—	—	—	⚠️	🔲 not targeted	⚠️	—	—
extension builds	✅	✅	🔲	✅	✅	✅ cargo-zigbuild	✅ cargo-zigbuild	✅ cargo-xwin	✅ cargo-xwin	✅ emscripten
wheel builds	✅	✅	❌ refuses by name	✅	✅	✅ manylinux_2_17_x86_64	✅ manylinux_2_17_aarch64	✅ win_amd64	✅ win_arm64	⚠️ by hand, not by build.py
symbols resolve	✅	✅	—	✅	✅ 118 names against the device framework	⚠️ weaker: ELF names only versioned imports	⚠️ same	✅ PE names every one	✅ PE names every one	✅ stub behaviour proven against the real host
dlopen + PyInit_ runs	—	—	—	—	—	—	—	—	—	✅
installs	✅	✅	—	✅	⚠️	✅	✅ pip matched the tag on real aarch64 Linux	✅	⚠️	✅ mounted, no wheel
import torch	✅	✅	—	✅	⚠️	✅	✅	✅	⚠️	✅
computes	✅	✅	—	✅	⚠️	✅	✅ glibc 2.17 and modern	✅	⚠️	✅
on PyPI 0.0.12a0	✅	✅	🔲	✅	✅	✅	🔲 built, not published	✅	🔲 built, not published	—
can be run here	✅	emulator	❌ no x86-64 emulator on Apple Silicon	simulator	❌	CI	✅ Docker, native aarch64	CI	❌	✅ Node

The two CUDA rows are the weakest cells in this table and are marked generously. computes is ⚠️ rather than 🔲 because the wiring is complete and only a GPU is missing — but strictly, ⚠️ reads "built, never executed" and nothing has been built either. This project's machine is an arm64 Mac with no nvcc, so no compiler has yet seen the CUDA-gated code; .github/workflows/build-cuda-wheel.yml is the one that will, and it has not run. docs/devices/CUDA.md §8 is the exact boundary and §6 is the procedure that would move these cells.

Linux and Windows now compute, and it is a run rather than an argument. A hosted runner is the machine this project does not have, so .github/workflows/verify-published-wheel.yml installs the published wheel from PyPI on ubuntu-latest and windows-latest at Python 3.13 and asks it to work. Both answered RESULT: ALL PASS — mm, nn.Linear, and the mixed-dtype promotion where the value and not just the label is at stake (int64(2049) - float16(1.0) is 2047.0) — and then both ran SmolLM2-135M through real transformers and produced text character-identical to macOS arm64. Every expected value is hardcoded from an arm64 run of the same source, so a disagreement would have localised to the platform rather than to the check.

The candle computes row for those two was stale, and this is the correction. It stood at ⚠️ — built, never executed — while the rows underneath it said the wheel installs, imports and computes on both, which cannot both be true: aten.mm.default on cpu is candle_core::Tensor::matmul, and there is no other compute path for that device. So mm sum 24.0 in run 34038982934 is candle computing on Linux x86-64 and on Windows amd64, and had been since that run went green on 2026-09-06. Both cells are now ✅ CI. The ⚠️ was left behind when the computes row moved and nobody came back up the column — the same mechanism docs/verification/AUDIT.md found behind six of eleven stale claims.

Linux aarch64 is the one column verified here rather than by CI, and it went further than CI does. Docker on this machine runs a native aarch64 Linux VM, so manylinux2014_aarch64 — CentOS 7, ldd (GNU libc) 2.17 — is the wheel's own tagged floor, not an approximation of it. The wheel was installed there and tools/ci/verify_published.py, the script both CI legs run, answered RESULT: ALL PASS over 31 checks. Then on a modern aarch64 Linux the same wheel was installed by bare distribution name from a local directory, so pip had to match manylinux_2_17_aarch64 against the machine to find any candidate at all, and SmolLM2-135M generated text character-identical to macOS arm64. Nothing here was timed (docs/platform/WHEELMATRIX.md §3.1).

Windows arm64 builds and stops there, and no machine in this project's reach can move it. The wheel is win_arm64, all 241 of its imports are attributed to a named DLL and 125 of them resolve against the ARM64 python3.dll — but there is no ARM64 Windows here, Docker's VM is Linux, and the CI job runs windows-latest, which is x86-64. A windows-11-arm runner would close it exactly the way ubuntu-latest closed Linux x86-64.

Android x86_64 refuses, and the missing piece is not the toolchain. The NDK's x86_64-linux-android21-clang runs here under Rosetta and the rust target is installed; what does not exist is an x86-64 Android CPython, which every wheel tag in this repository is derived from. python-build-standalone publishes 871 assets in the release the other four distributions come from and not one is Android. And the result could not be checked if it were built: this machine's emulator ships only qemu/darwin-aarch64, so Apple Silicon runs aarch64 guests and nothing else. --target android-x86_64 therefore refuses by name, with that reason, rather than being dropped from the registry and looking like a target nobody considered.

Windows had moved once before on a user's report rather than our own run, and that report could not carry computes: they installed 0.0.5a0 with uv, import torch succeeded, and transformers carried them 655 lines into modeling_rope_utils.py before a missing meta kernel — since fixed — but everything on that path ran on the meta device, which by construction computes nothing.

The same report settled something else the table has no row for. Their interpreter was Python 3.14 and the wheel is cp313-abi3. One binary per platform loading on 3.13 and every later CPython is the property five published wheels rest on, and until then it had only been argued.

The last row is why the columns differ, and the iOS column had to become two. The simulator computes: verify_ios_sim.py boots one, unpacks the published wheel into an iOS CPython's site-packages, and gets aten.mm, x + x and an nn.Linear forward, with platform.system() answering iOS. That runs on every release build and in CI. It was previously reported as ⚠️ along with the device, which understated it.

The device is the one platform with no way to run, and CI cannot close it. A hosted macOS runner has a simulator, not an iPhone — the same rung, on somebody else's machine — and the simulator runs on the host kernel, which its own output says: uname().version is this Mac's. Same instruction set, different Mach-O platform, separate artefact. So the device column stays at symbols resolve until a physical device runs it.

WASM is the exception, and it has now been executed. A complete emsdk with emcc and a bundled Node 24 sits in this machine's cache — command -v node finds nothing only because it is not on PATH, which an earlier draft of this line published as "no node on this machine". Under a real Pyodide the extension loads, import torch returns 2.13.0 from the vendored tree, and a @ b and an nn.Linear forward match a host build. Two things keep it short of the others: Pyodide ships CPython 3.14, not 3.13, so the module is tied to one interpreter rather than to an abi3 floor — and torch/__init__.py imports torch.multiprocessing, which a browser sandbox cannot supply, so that import is stubbed by the harness rather than solved. A WASM wheel has been built by hand, installed into Pyodide 314.0.6 and imported; build.py does not build one, because verify_cross.py reads ELF and Mach-O symbol tables that a wasm module does not have, and a target this repo cannot check would ship unchecked (docs/platform/WASM.md §9).

Devices

device	macOS	Android	iOS	Linux	Windows	WASM	what it is
cpu	✅	✅	⚠️	⚠️	⚠️	✅	the only device that holds a tensor
meta	✅	✅	⚠️	⚠️	⚠️	🔲	shape and dtype, no storage
mps	✅	—	🔲	—	—	—	candle's Metal backend, on. An op whose kernel would compute on the CPU under an mps label is refused at the door, naming the op — 54 of them, listed by _C._shim_mps_host_readback_ops() (docs/devices/MPS.md)
vulkan	✅	❌	—	🔲	🔲	—	eighteen ops by name through real VkBuffers (eleven compute kernels); an MLP forwards; every other op refuses naming itself
NNAPI · CoreML	—	❌	❌	—	—	—	needs the graph path, blocked at decomposition
cuda	—	—	—	⚠️	⚠️	—	Wired, never run. One resolve() arm, because Device::Cuda is already a variant of candle's enum — the mps shape, not the vulkan one, so no kernel of ours. Off unless built with --cfg torch_c_cuda, which reaches Linux and Windows only. When unavailable it names which of not_built / no_driver / no_device / wrong_arch / unclassified it is. Nothing has compiled it (this machine has no nvcc) and nothing has run it (docs/devices/CUDA.md)
WebGPU	—	—	—	—	—	🔲	the only accelerator a browser offers

dtypes on cpu

11 of 46 storable, and the same 11 on both platforms measured — Android was probed on the device rather than inferred from the host.

dtype	macOS	Android	iOS	Linux · Windows · WASM	arithmetic path
float32	✅	✅	⚠️	⚠️ · ⚠️ · 🔲	macOS: AMX via Accelerate · Android: NEON gemm, 88% of core peak
float64	✅	✅	⚠️	⚠️ · ⚠️ · 🔲	gemm. Mixes with the other float and integer dtypes, on add, sub, the six comparisons, max/min, bitwise_or, where, cat and stack — result dtype and value bit-identical to upstream over a 9×9 grid, which is not the same thing: upstream casts each operand to the common dtype before the accumulator, so int64(2049) - float16(1.0) is 2047.0 and not 2048.0 (docs/numerics/PROMOTE.md). mm, matmul, bmm, convolution and SDPA still refuse a mixed pair, and so does upstream
bfloat16 · float16	✅	✅	⚠️	⚠️ · ⚠️ · 🔲	widened to f32 in registers, accumulated, narrowed once — upstream's rule. Prefill is 1.19x float32 here and 2.3x faster than upstream's own bfloat16; decode still materialises the widened weight (docs/perf/DTYPE_PERF.md)
bool uint8 uint32 int16 int32 int64	✅	✅	⚠️	⚠️ · ⚠️ · 🔲	integer kernels
float8_e4m3fn	⚠️	⚠️	⚠️	⚠️ · ⚠️ · 🔲	it no longer hangs — the hang was infinite recursion in candle's own with_dtype! for this type, which release-mode tail-call optimisation collapses into a bare jmp to itself, so it span the CPU without ever overflowing the stack. Comparison, tolist, item and matmul refuse by name instead. It diverges the other way too: upstream ships mul and abs for this dtype and refuses add/sub/div/neg/exp/sum/mean, **and this build now refuses the same set, in upstream's own kernel wording. A first probe found seven; enumerating all 197 ops found 114 divergent — 48 computed and 27 hung where upstream refuses (docs/numerics/FLOAT8B.md). In the golden suite now — the exclusion reason was that construction hung on both sides, which stopped being true
int8 qint8 quint8	❌	❌	❌	❌	candle's DType has no I8: the tensor cannot be created. Adding it would buy the storage type and not the speed — candle has no int8 matmul either, and its quantisation is QTensor/GgmlDType, a separate system that Tensor/DType never sees, so teaching aten.mm a new element type does not reach the fast kernel. int8 inference is here, as module replacement rather than as a dtype. It is reached at load time through the slot transformers provides for it — from_pretrained(name, quantization_config=TorchnativeConfig("q8_0")), a registered HfQuantizer — and the leaves are swapped before the weights land, so the dense model is never assembled: peak RSS for SmolLM2-135M is 924 MB against 1231 dense and 1337 quantising afterwards. dtype=torch.int8 in that same call is closed by transformers itself, before any of this runs (docs/graph/HFQUANT.md)
the other 35	❌	❌	❌	❌	complex, other float8, 4-bit — refuse by name

The last two rows are ❌ everywhere rather than 🔲, because the cause is in candle's type system and does not vary by platform.

Quantisation — beside the dtype system, not inside it

candle keeps quantisation in a separate QTensor type, which is why int8 being unstorable does not block it. Reached through torchnative.quant, which swaps nn.Linear.

format	macOS	Android	iOS	Linux · Windows · WASM	note
Q8_0	✅	✅	⚠️	⚠️ · ⚠️ · 🔲	lossless on integer operands — bit-identical to a dense linear
Q4_0	✅	✅	⚠️	⚠️ · ⚠️ · 🔲	29.5% logit RMS on SmolLM2; degrades generation
Q4K	✅	✅	⚠️	⚠️ · ⚠️ · 🔲	a k-quant, needing k % 256 — a model constraint, not a platform one

All three were measured on macOS and on the Android device with the same probe. SmolLM2 cannot use the k-quants because its layers are 576 wide and 576 is not a multiple of 256 — that is about the model, and an earlier draft of this table wrongly put it in the platform column.

Android Q4K is 1.60× f32 at prefill as shipped, and 3.29× with +dotprod — which cannot be turned on, candle having no runtime dispatch and ARMv8.0 devices no sdot (docs/graph/QUANT.md).

Linux, Windows and WASM

Linux x86_64 crosses four of six layers (docs/platform/LINUX.md). One thing blocks it, and it is not the linker — rust-lld ships with rustup and links ELF fine. It is that x86_64-unknown-linux-gnu is the one target rustup ships no glibc stubs for, and that candle → tokenizers → onig → onig_sys is a C crate, so the build stops at failed to find tool "x86_64-linux-gnu-gcc" before linking is even reached. cargo-zigbuild supplies all of it and is not installed; that is a decision, not an oversight.

Windows x86_64 has its CPython distribution and nothing else yet.

WASM runs. Under Emscripten and the Node in this machine's emsdk, candle computes a quantised matmul to 511.96875 — bit-identical to the host, the same quantisation error rather than a round number agreeing — and dlopen loads our own cdylib, whose PyInit_ executes and returns a module definition (docs/platform/WASM.md §7). The onig subtree drops out there, so the dependency count falls 129 → 80.

What it costs is abi3. Pyodide pins CPython 3.13, 3.14 and 3.15 to Emscripten 4.0.9, 5.0.3 and 6.0.5 — three releases, three compilers — so WASM would be one binary per CPython feature release rather than one per platform. That is a different distribution model from the other five, not a variation on it. WASI is separately blocked: no dlopen, so torch._C cannot be a wheel there at all — but WASI was never the route, and Pyodide, which is, has dynamic linking. And PEP 783 forbids -pthread, so the honest line is scalar and single-threaded — simd128 is off because candle's own WASM SIMD backend does not compile.

It is absent from the matrix on purpose: that table is a kernels backend matrix, and kernels has no wasm backend — the same gap it already records for vulkan.

---

Verification

Correctness here means agreeing with upstream PyTorch, so the strategy is comparison rather than assertion.

Golden comparison	Every operator runs on both upstream torch and this shim, compared on value, shape and dtype. It has caught a float16 GEMM accumulating in float16 where torch accumulates in float32, cumsum routed through the wrong kernel, and integer overflow where torch refuses.
The harness tests itself	--self-test injects a fault shaped like a plausible misimplementation at each comparator and fails if the comparator accepts it — 11 comparators × 11 fault modes, with any comparator never exercised reported as failure. It found that the previous fault injection reached exactly one case out of 1781.
Tokens are not enough	A wrong gelu approximation produced identical tokens while logits differed by 5.9e-04. End-to-end tests compare logits too, with a tolerance measured to sit between normal float32 noise and that failure.

sh rust/torch_c/pytests/run.sh                  # smoke tests + harness self-test
python tools/golden/compare.py                  # golden comparison against upstream
python rust/torch_c/pytests/verify_schemas.py   # signature tables vs upstream

---

Roadmap

The next milestone is the device abstraction, because everything waits on it — a distributed rank needs a device to point at, and every accelerator attaches there.

torchnative.nn.federated    rounds · client selection · aggregation · dropout
  └ torch.distributed       ProcessGroup · collectives (transport)
      └ backends            ours, via register_backend
          └ devices         CPU · Metal · Vulkan · NPU

	what is measured today
Device abstraction	Done. torch.device, per-device dispatch, and a Repr arm per device. Everything below attached here.
Metal	On, computing on the real GPU (Apple M1, candle's Metal backend), Apple targets only. An mps tensor is an ordinary candle tensor, so no kernel had to be taught it (docs/devices/VULKAN3.md) — which is also why it needs a gate the Vulkan representation does not: the ops whose kernels read the tensor back to the host are refused by name, aten._softmax.default among them, so a transformer does not forward on mps (docs/devices/MPS.md).
Vulkan	Wired and computing, through real VkBuffers on this host — a fourth arm of tensor::Repr outside candle entirely, which is what makes a silent CPU fallback unrepresentable rather than merely avoided. Eighteen ops by name -- eleven SPIR-V compute kernels and seven metadata ops -- chosen by tracing what a real forward pass dispatches, and a whole nn.Sequential(Linear, ReLU, Linear) forwards on the GPU with zero host readbacks. A transformer does not: native_layer_norm, _softmax, gelu, embedding and bmm are on that trace and all still refuse. Performance still needs a phone (docs/devices/VULKAN4.md).
CUDA	Wired, and that is the whole claim. Device::Cuda is already a variant of candle's closed enum, so a cuda tensor is an ordinary candle tensor and not one kernel had to be written — the same asymmetry that put mps before vulkan. Target-scoped and off by default: --cfg torch_c_cuda on Linux/Windows only, structurally unreachable from Android, iOS and wasm. Unavailability is a named refusal — not_built, no_driver, no_device, wrong_arch, unclassified — and the readback gate that mps needed applies unchanged, from the same derived list, and matters more there because CUDA implements the f64 that made those kernels loud on Metal. Nothing has been compiled with CUDA on and nothing has run on a GPU: this project's only machine has no nvcc. A CI job builds it and has not run; docs/devices/CUDA.md §6 is the procedure for a machine with a GPU and §8 is the honest boundary.
torch.distributed	world_size >= 3 runs, over loopback TCP in a star with the hub at rank 0. Only allreduce(op=SUM) is implemented; every other collective and every other reduce op refuses by name (docs/distributed/FEDERATED4.md, docs/distributed/TRANSPORT.md).
NPU	The capture layer is built and a whole model lowers through it — prims folded back to aten, BatchNorm fused into the preceding convolution, nothing left outside NNAPI's op set for mobilenet_v2. CoreML executes; the NNAPI blob is structurally validated and has never met an NPU (docs/graph/NPU.md).
Eager training	loss.backward() and an optimizer step work and match upstream to one float32 ulp on a small nn.Sequential. Not a milestone that is finished: no transformer has been trained through it, and there is no convolution backward rule (docs/training/BACKWARD9.md).
torch.compile	Not on this roadmap. docs/graph/COMPILE.md recommends refusing it by name, permanently: PEP 523 frame evaluation needs CPython internals that cannot coexist with the limited API in one extension, and abi3 is what makes one binary per platform serve 3.13 and later. torch.export is the direction instead, and it is not implemented (docs/graph/COMPILE.md).

This table records what has been measured, not what is planned. Where it disagrees with a docs/ file, the file is the measurement and this is the summary. It said torch.distributed was coming "from world_size = 1 upward", NPU "needs a capture layer" and Metal was "disabled here" for some days after all three had landed — a roadmap is a progress record, and a stale progress record misleads in the one direction a reader cannot check.

---

Install

pip install torchnative

Every published version is a pre-release, so if your resolver is configured to skip those, ask for one by name: pip install --pre torchnative.

0.0.12a0 ships five platform wheels, all cp313-abi3 — one binary per platform, loadable by CPython 3.13 and every later release. Each carries the _C extension and the vendored upstream tree, so import torch resolves to this build.

They are not all verified to the same depth, and the table says which is which.

wheel	built	installed	import torch	computes
macosx_11_0_arm64	✅	✅	✅	✅
android_21_arm64_v8a	✅	✅	✅	✅
ios_12_0_arm64_iphoneos	✅	—	—	—
manylinux_2_17_x86_64	✅	—	—	—
win_amd64	✅	—	—	—

The iOS simulator wheel is published, and this sentence used to say the opposite — that it was "deliberately not published" because a resolver reaching it would be trapped. PyPI has carried it since 0.0.2a0; the two iOS wheels differ by platform tag (iphonesimulator against iphoneos) and pip selects on that, so the trap does not exist. Corrected rather than left, because it is the kind of claim nobody re-reads.

Linux and Windows were in that position and are not any more: CI installs the published wheel on ubuntu-latest and windows-latest and both compute, matching macOS arm64 character for character on a real SmolLM2 generation. The green runs installed the version the workflow defaults to, which is what tools/ci/verify_published.py was written against; checks added for a later release skip themselves by name on an older wheel rather than failing the platform.

What follows is the artefact-level check that used to be all there was, and it still runs — it catches a broken wheel before anything is uploaded. Every import in the Linux wheel resolves, and every import in the Windows one is attributed to a naming DLL, which is the stronger of the two checks because PE records a DLL per import where ELF records only versioned ones (docs/platform/LINUX.md, docs/platform/WINDOWS.md).

macOS is checked in a clean virtualenv and Android on a device, unpacked into its CPython's site-packages — in both, torch.__file__ lands inside the install, aten.mm returns the right answer and an nn.Linear forward runs (docs/platform/WHEEL.md §7).

[!IMPORTANT] The iOS wheel has never been executed. What is verified is everything short of running it: its 222 undefined symbols all resolve against the device Python.framework and the iOS SDK, checked through the two-level namespace bindings dyld itself uses, and every file in it outside the extension is byte-identical to the simulator wheel, which does import and compute. What is not verified is the load itself, @rpath resolution inside a real app bundle, and code signing — none of which can be answered without a device (docs/platform/IOS.md).

If you run it on a phone, we would like to hear either way.

[!NOTE] 0.0.1a0 is still on PyPI and does not work — it is py3-none-any and carries the torchnative skeleton alone, no _C and no torch, so it installs cleanly and then fails to import. Ask for 0.0.12a0 or later.

There is no source distribution. Building needs a Rust toolchain and a vendoring step that pip cannot drive, so an sdist would install and then fail; the recipe is below instead.

Building from source

Requires a Rust toolchain and CPython 3.13+.

bash vendor/vendor_torch.sh     # assemble the vendored torch tree
bash vendor/install_shim.sh     # build the extension and install it

Building a wheel

Additionally requires pip, setuptools and wheel in the building interpreter, and a C compiler for the empty libtorch_global_deps (see docs/platform/WHEEL.md §3.2).

bash vendor/vendor_torch.sh
bash vendor/install_shim.sh
python tools/wheel/build.py                            # -> dist/*.whl
python tools/wheel/verify.py dist/torchnative-*.whl    # clean venv, real import

verify.py is the part that matters: it installs into a throwaway virtualenv and asserts that torch.__file__ resolves inside it. A check that lets the development tree answer proves nothing about the wheel.

Cross-compilation is documented in docs/platform/RUST_CROSSBUILD.md, including the PyO3 configuration iOS needs in order not to link libpython.

---

Repository layout

torchnative/     the Python library
rust/torch_c/    the torch._C replacement (Rust · PyO3 · candle)
tools/golden/    the upstream comparison harness
tools/wheel/     build a platform wheel, and prove it installs (docs/platform/WHEEL.md)
vendor/          scripts that assemble the vendored torch tree (not checked in)
docs/            design, measurements, and the reasoning behind open decisions

docs/ is written to be read. It records what was measured, what was assumed, and where an earlier conclusion turned out to be wrong — corrections are left visible rather than edited away. Start with DESIGN.md; SURFACE_HONESTY.md and HARNESS.md show the standard the rest aims for.

---

Related

• PythonMultiplatform — embeds CPython 3.13 into Kotlin Multiplatform; the deployment target for this library
• pypackpack — the build and bundling tool
• Hugging Face kernels — the fused-kernel contract this adopts, with resolution moved from runtime download to build time, since downloading executable code is not permitted on every target platform

---

License

Apache-2.0 — see LICENSE.

The repository and the wheel are not the same thing, and they carry different licences. This repository contains only this project's code: the vendored PyTorch tree is assembled at build time (vendor/vendor_torch.sh) and is not redistributed here. A platform wheel is different — it carries upstream PyTorch's entire Python tree with torch._C replaced, so most of the files in an installed torchnative are upstream's, under upstream's licences.

Upstream's terms are not one licence. pyproject.toml's license field is torch 2.13.0's own License-Expression, verbatim:

Apache-2.0 AND Apache-2.0 WITH LLVM-exception AND BSD-2-Clause
AND BSD-3-Clause AND BSL-1.0 AND MIT

That expression is unchanged by this project's move from MIT to Apache-2.0, because it already contained both terms — the field cannot express which term is this project's. The upstream licence texts themselves ride along: tools/wheel/build.py injects torch's dist-info, third-party notices included.