This release is a pre-release and may not be stable for production use.
Gemma XDNA2
Experimental full-model Gemma 3 1B IT inference on AMD XDNA2, using the accepted
full26-c32768-selected PDI from the restored September 2026 research archive.
Version 0.1.0a2 is prepared locally; it has not been published to PyPI or the Hub.
The wheel includes a C++ XRT session object, seven Cython AOT host modules,
the frozen PDI execution set, a private XRT 2.23.0 runtime and the Apache-licensed
userspace XDNA shim. No own Python source or bytecode is distributed. It does not install a kernel driver,
firmware, compiler, or system XRT. Weights and tokenizer files are in a separate
Hugging Face model bundle, staged in release/huggingface.
Run the local release
Use CPython 3.12 on Linux x86_64. The tested host is Ubuntu 26.04.1, kernel
7.0.0-43-generic, /dev/accel/accel0, RyzenAI-npu6/aie2p, firmware 1.1.2.64.
The PyPI wheel is audited as manylinux_2_39_x86_64. It requires glibc >= 2.39
and libstdc++ providing GLIBCXX_3.4.32 and CXXABI_1.3.15. The manylinux tag
describes user-space ABI compatibility; other NPU/driver combinations remain
unqualified.
uv venv --python 3.12.16 .venv-run
uv pip install --python .venv-run/bin/python \
dist/gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl
.venv-run/bin/gemma-xdna2 doctor --probe
.venv-run/bin/gemma-xdna2 verify release/huggingface
.venv-run/bin/gemma-xdna2 generate --model release/huggingface \
--prompt 'What is the capital of France?' --max-tokens 24 --json
The user must have read/write access to the NPU device and at least 2 GiB of
memory-lock allowance. doctor reports both. A hard limit below 2 GiB needs an
administrator to raise it using prlimit for the launching process or a PAM
limits.d entry for future logins. This project never changes those settings
automatically. On the recovery host, odroid now has a persistent 4 GiB PAM
limit in /etc/security/limits.d/90-gemma-xdna2.conf; the current Codex parent retains an older 8 MiB hard limit. One-shot jobs
launched through the existing cron PAM configuration obtain the configured
4 GiB limit, remove their temporary cron entry and execute the validation.
A fresh SSH login itself was not exercised.
After publication, download using the exact Hub commit:
gemma-xdna2 download --repo yongseungyu/Gemma-3-1B-IT-XDNA2-Q4_1 \
--revision <published-40-character-commit> --output ./gemma-model
gemma-xdna2 generate --model ./gemma-model --prompt 'Hello' --max-tokens 32
The repository name above is proposed. Its commit is assigned only when it is published. Authentication uses the standard Hugging Face environment or token store; no credential is bundled.
Execution and numerical contract
All 26 transformer layers, embedding, normalization, RoPE, attention, MLP, residuals, KV writes and full-vocabulary head run on the NPU. The host writes parameters, starts/waits, reads logits, checks finiteness and chooses greedy argmax. There is one persistent hardware context and one initial weight load. Requests are serialized through a per-user NPU lock. There is no CPU fallback.
Projection weights use the archived native Q4_1 packet layout, with BF16 scale and minimum per group. Original BF16 embedding/norms are retained; activations and KV use the kernel's int8-g32 contract. This is a custom artifact format and is consumed by this package, independently of Transformers model execution.
Prompt tokens are ingested through sequential full-model NPU steps. Optimized
batched prefill and native llama.cpp integration are future work. The image
has 32768-token allocation capacity. The 128--32K execution/timing sweep and
strict KV diagnostic findings are documented in docs/context-sweep.md;
complete long-context quality qualification is still pending. Generation is deterministic greedy
decoding; sampling is not implemented.
See validation, rebuild instructions and
release preparation. Model quality remains REPORT_ONLY.
Licenses
Project runtime software and restored kernels retain Apache-2.0; bundled components retain their Apache/BSD/MIT/Boost licenses and notices. Only compiled project code is distributed; this does not withdraw those license grants or make binaries impossible to inspect. See BINARY_DISTRIBUTION.md. The separate Gemma-derived weights are governed by the Gemma Terms of Use. The model bundle contains the terms, use policy, required notice, source attribution and modification log.
Metadata
Release files for gemma-xdna2 0.1.0a2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl | CPython 3.12 | CPython 3.12 | Linux glibc 2.39+ x86-64 | Details for gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl |
Release files / gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl
| Download URL | gemma_xdna2-0.1.0a2-cp312-cp312-manylinux_2_39_x86_64.whl |
|---|---|
| Size | 2.9 MB |
| Tags | CPython 3.12 Linux glibc 2.39+ x86-64 |
|
SHA-256 checksum How to use checksums |
00a8b8897b2bc3838a490f032ed4bec3c4c5492135a199fff88609ce72b8fcfe
|
|
BLAKE2b-256 checksum How to use checksums |
7e4631deec2284aae594f157b636f1d031808cb75b87e7224b833a18b687f64c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.16
|
Security for gemma-xdna2 0.1.0a2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.