This release is a pre-release and may not be stable for production use.
onnx-genai
ONNX Runtime-backed text generation for Python.
onnx-genai exposes a small, high-level generation API implemented on
ONNX Runtime. It is the ONNX Runtime-compatible
counterpart of nxrt.genai: both packages present the same Engine /
GenerateResult API, but onnx_genai runs on ONNX Runtime while nxrt.genai
runs on the native nxrt runtime.
import onnx_genai
engine = onnx_genai.Engine.from_dir("path/to/model_dir")
result = engine.generate("Hello, world!", max_tokens=64, temperature=0.8)
print(result.text)
# Streaming
def on_token(text, token_id, finish_reason):
print(text, end="", flush=True)
engine.generate_stream("Tell me a story", on_token, max_tokens=128)
The model directory must contain the ONNX graph(s), tokenizer.json, and model
metadata (inference_metadata.yaml or genai_config.json).
Wheels
Distributed as stable-ABI (abi3) wheels tagged cp310-abi3, so a single wheel
per platform loads on CPython 3.10 and newer. ONNX Runtime is not bundled:
onnx_genai loads libonnxruntime from the installed onnxruntime (CPU) or
onnxruntime-gpu (CUDA) wheel at import time, so it uses whichever execution
providers you installed.
Related packages
nxrt— the low-level nxrt runtime (inference sessions, eager ops) plusnxrt.genai, the same generation API backed by the native runtime.onnx-genai-server— theonnx-genaicommand-line tool, including the OpenAI-compatible server (onnx-genai serve).
Metadata
Release files for onnx-genai 0.1.0.dev6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| onnx_genai-0.1.0.dev6.tar.gz | 12.8 MB | With provenance | Details for onnx_genai-0.1.0.dev6.tar.gz |
Built distributions (wheels)
| File | Actions Reset | |||
|---|---|---|---|---|
| onnx_genai-0.1.0.dev6-cp310-abi3-win_arm64.whl | CPython 3.10 | abi3 | Windows ARM64 | Details for onnx_genai-0.1.0.dev6-cp310-abi3-win_arm64.whl |
| onnx_genai-0.1.0.dev6-cp310-abi3-win_amd64.whl | CPython 3.10 | abi3 | Windows x86-64 | Details for onnx_genai-0.1.0.dev6-cp310-abi3-win_amd64.whl |
| onnx_genai-0.1.0.dev6-cp310-abi3-manylinux_2_28_x86_64.whl | CPython 3.10 | abi3 | Linux glibc 2.28+ x86-64 | Details for onnx_genai-0.1.0.dev6-cp310-abi3-manylinux_2_28_x86_64.whl |
| onnx_genai-0.1.0.dev6-cp310-abi3-macosx_11_0_arm64.whl | CPython 3.10 | abi3 | macOS 11.0+ ARM64 | Details for onnx_genai-0.1.0.dev6-cp310-abi3-macosx_11_0_arm64.whl |
Total release size: 50.1 MB
Release files / onnx_genai-0.1.0.dev6.tar.gz
| Download URL | onnx_genai-0.1.0.dev6.tar.gz |
|---|---|
| Size | 12.8 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
12f9b54a42b7f7cbf4dd11e60253a3cba2c28aab7443d667e2bb48984c44ed4c
|
|
BLAKE2b-256 checksum How to use checksums |
24d7ffd103c744961d317eef59016a1441006c4a41cea0105b16830e12103ff3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 10, 2026.
Transparency logRelease files / onnx_genai-0.1.0.dev6-cp310-abi3-win_arm64.whl
| Download URL | onnx_genai-0.1.0.dev6-cp310-abi3-win_arm64.whl |
|---|---|
| Size | 8.8 MB |
| Tags | CPython 3.10 Windows ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
33c462064ade56210db33840d15efc61d01eb99c9403e0b1bf115e285041a78f
|
|
BLAKE2b-256 checksum How to use checksums |
8d75df47a2e90eaf222f9e4b5ee0558da1292411baf2c3b7b10e32acd6c93421
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 10, 2026.
Transparency logRelease files / onnx_genai-0.1.0.dev6-cp310-abi3-win_amd64.whl
| Download URL | onnx_genai-0.1.0.dev6-cp310-abi3-win_amd64.whl |
|---|---|
| Size | 10.0 MB |
| Tags | CPython 3.10 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
60686a410fc0edf0721af4f86c0df0f7432da3ae4c7477a7b925fa92faf51c18
|
|
BLAKE2b-256 checksum How to use checksums |
7a788baf984e69c612fad4cf48ffb9d70dbed14bcc13cff0e25e40e10dbcd7e2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 10, 2026.
Transparency logRelease files / onnx_genai-0.1.0.dev6-cp310-abi3-manylinux_2_28_x86_64.whl
| Download URL | onnx_genai-0.1.0.dev6-cp310-abi3-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 10.0 MB |
| Tags | CPython 3.10 Linux glibc 2.28+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
ea27400c4bd51b6c2daa26e073f959d6c00665070e8fe0c7a3dbcfe50de21d4e
|
|
BLAKE2b-256 checksum How to use checksums |
5019b66bfc0b8abf478d3c92dd3603eb7de0d8f1370d29632ee4df7621023e49
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 10, 2026.
Transparency logRelease files / onnx_genai-0.1.0.dev6-cp310-abi3-macosx_11_0_arm64.whl
| Download URL | onnx_genai-0.1.0.dev6-cp310-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 8.5 MB |
| Tags | CPython 3.10 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
ed77d07a22f2d0265d89fdd5c857f3fd4f9df20d8f499d044cc121638cf959ab
|
|
BLAKE2b-256 checksum How to use checksums |
28fbca1e715daf32663c5ae159f82adf4efa2e9afa71510d84da5ed5aa676ce0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 10, 2026.
Transparency logSecurity for onnx-genai 0.1.0.dev6
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 5 files in this release have been uploaded with provenance metadata.
Attestation for onnx_genai-0.1.0.dev6.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 10, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.