Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

Gemma XDNA2

Experimental full-model Gemma 3 1B IT inference on AMD XDNA2, using the accepted full26-c32768-selected PDI from the restored September 2026 research archive. Version 0.1.0a2 is prepared locally; it has not been published to PyPI or the Hub.

The wheel includes a C++ XRT session object, seven Cython AOT host modules, the frozen PDI execution set, a private XRT 2.23.0 runtime and the Apache-licensed userspace XDNA shim. No own Python source or bytecode is distributed. It does not install a kernel driver, firmware, compiler, or system XRT. Weights and tokenizer files are in a separate Hugging Face model bundle, staged in release/huggingface.

Run the local release

Use CPython 3.12 on Linux x86_64. The tested host is Ubuntu 26.04.1, kernel 7.0.0-43-generic, /dev/accel/accel0, RyzenAI-npu6/aie2p, firmware 1.1.2.64. The PyPI wheel is audited as manylinux_2_39_x86_64. It requires glibc >= 2.39 and libstdc++ providing GLIBCXX_3.4.32 and CXXABI_1.3.15. The manylinux tag describes user-space ABI compatibility; other NPU/driver combinations remain unqualified.

uv venv --python 3.12.16 .venv-run
uv pip install --python .venv-run/bin/python \
  dist/gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl
.venv-run/bin/gemma-xdna2 doctor --probe
.venv-run/bin/gemma-xdna2 verify release/huggingface
.venv-run/bin/gemma-xdna2 generate --model release/huggingface \
  --prompt 'What is the capital of France?' --max-tokens 24 --json

The user must have read/write access to the NPU device and at least 2 GiB of memory-lock allowance. doctor reports both. A hard limit below 2 GiB needs an administrator to raise it using prlimit for the launching process or a PAM limits.d entry for future logins. This project never changes those settings automatically. On the recovery host, odroid now has a persistent 4 GiB PAM limit in /etc/security/limits.d/90-gemma-xdna2.conf; the current Codex parent retains an older 8 MiB hard limit. One-shot jobs launched through the existing cron PAM configuration obtain the configured 4 GiB limit, remove their temporary cron entry and execute the validation. A fresh SSH login itself was not exercised.

After publication, download using the exact Hub commit:

gemma-xdna2 download --repo yongseungyu/Gemma-3-1B-IT-XDNA2-Q4_1 \
  --revision <published-40-character-commit> --output ./gemma-model
gemma-xdna2 generate --model ./gemma-model --prompt 'Hello' --max-tokens 32

The repository name above is proposed. Its commit is assigned only when it is published. Authentication uses the standard Hugging Face environment or token store; no credential is bundled.

Execution and numerical contract

All 26 transformer layers, embedding, normalization, RoPE, attention, MLP, residuals, KV writes and full-vocabulary head run on the NPU. The host writes parameters, starts/waits, reads logits, checks finiteness and chooses greedy argmax. There is one persistent hardware context and one initial weight load. Requests are serialized through a per-user NPU lock. There is no CPU fallback.

Projection weights use the archived native Q4_1 packet layout, with BF16 scale and minimum per group. Original BF16 embedding/norms are retained; activations and KV use the kernel's int8-g32 contract. This is a custom artifact format and is consumed by this package, independently of Transformers model execution.

Prompt tokens are ingested through sequential full-model NPU steps. Optimized batched prefill and native llama.cpp integration are future work. The image has 32768-token allocation capacity. The 128--32K execution/timing sweep and strict KV diagnostic findings are documented in docs/context-sweep.md; complete long-context quality qualification is still pending. Generation is deterministic greedy decoding; sampling is not implemented.

See validation, rebuild instructions and release preparation. Model quality remains REPORT_ONLY.

Licenses

Project runtime software and restored kernels retain Apache-2.0; bundled components retain their Apache/BSD/MIT/Boost licenses and notices. Only compiled project code is distributed; this does not withdraw those license grants or make binaries impossible to inspect. See BINARY_DISTRIBUTION.md. The separate Gemma-derived weights are governed by the Gemma Terms of Use. The model bundle contains the terms, use policy, required notice, source attribution and modification log.

Metadata

Release files for gemma-xdna2 0.1.0a2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for gemma-xdna2 0.1.0a2
File Interpreter ABI Platform Actions
gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.39+ x86-64 Details for gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl

Release files / gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl

Download URL gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl
Size 2.9 MB
Tags CPython 3.12 Linux glibc 2.39+ x86-64
SHA-256 checksum
How to use checksums
00a8b8897b2bc3838a490f032ed4bec3c4c5492135a199fff88609ce72b8fcfe
BLAKE2b-256 checksum
How to use checksums
7e4631deec2284aae594f157b636f1d031808cb75b87e7224b833a18b687f64c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.16

Security for gemma-xdna2 0.1.0a2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page