Pre-release

This release is a pre-release and may not be stable for production use.

onnx-genai

ONNX Runtime-backed text generation for Python.

onnx-genai exposes a small, high-level generation API implemented on ONNX Runtime. It is the ONNX Runtime-compatible counterpart of nxrt.genai: both packages present the same Engine / GenerateResult API, but onnx_genai runs on ONNX Runtime while nxrt.genai runs on the native nxrt runtime.

import onnx_genai

engine = onnx_genai.Engine.from_dir("path/to/model_dir")

result = engine.generate("Hello, world!", max_tokens=64, temperature=0.8)
print(result.text)

# Streaming
def on_token(text, token_id, finish_reason):
    print(text, end="", flush=True)

engine.generate_stream("Tell me a story", on_token, max_tokens=128)

The model directory must contain the ONNX graph(s), tokenizer.json, and model metadata (inference_metadata.yaml or genai_config.json).

Wheels

Distributed as stable-ABI (abi3) wheels tagged cp310-abi3, so a single wheel per platform loads on CPython 3.10 and newer. ONNX Runtime is not bundled: onnx_genai loads libonnxruntime from the installed onnxruntime (CPU) or onnxruntime-gpu (CUDA) wheel at import time, so it uses whichever execution providers you installed.

Related packages

nxrt — the low-level nxrt runtime (inference sessions, eager ops) plus nxrt.genai, the same generation API backed by the native runtime.
onnx-genai-server — the onnx-genai command-line tool, including the OpenAI-compatible server (onnx-genai serve).

See https://github.com/justinchuby/onnx-genai.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

onnx_genai-0.1.0.dev4.tar.gz (3.2 MB view details)

Uploaded Jul 24, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

onnx_genai-0.1.0.dev4-cp310-abi3-win_arm64.whl (4.6 MB view details)

Uploaded Jul 24, 2026 CPython 3.10+Windows ARM64

onnx_genai-0.1.0.dev4-cp310-abi3-win_amd64.whl (5.0 MB view details)

Uploaded Jul 24, 2026 CPython 3.10+Windows x86-64

onnx_genai-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl (5.1 MB view details)

Uploaded Jul 24, 2026 CPython 3.10+manylinux: glibc 2.28+ x86-64

onnx_genai-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl (4.5 MB view details)

Uploaded Jul 24, 2026 CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file onnx_genai-0.1.0.dev4.tar.gz.

File metadata

Download URL: onnx_genai-0.1.0.dev4.tar.gz
Upload date: Jul 24, 2026
Size: 3.2 MB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai-0.1.0.dev4.tar.gz
Algorithm	Hash digest
SHA256	`798f9b1fb714ddb8911e838e9f2fb9a559a5ced4cae373a922acece355c51812`
MD5	`b91c436f7f7e0990b472f63d39873088`
BLAKE2b-256	`8e6425b98c132669c73385956a2b63af9fab488827711dfb47939447e30e83c7`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai-0.1.0.dev4.tar.gz:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai-0.1.0.dev4.tar.gz
- Subject digest: 798f9b1fb714ddb8911e838e9f2fb9a559a5ced4cae373a922acece355c51812
- Sigstore transparency entry: 2238558378
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

File details

Details for the file onnx_genai-0.1.0.dev4-cp310-abi3-win_arm64.whl.

File metadata

Download URL: onnx_genai-0.1.0.dev4-cp310-abi3-win_arm64.whl
Upload date: Jul 24, 2026
Size: 4.6 MB
Tags: CPython 3.10+, Windows ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai-0.1.0.dev4-cp310-abi3-win_arm64.whl
Algorithm	Hash digest
SHA256	`a493e9597ee01d17cde5ebc1b5c5ba580370d470c6a6122de2cb1ee1d54efe87`
MD5	`f37c128be0d9255980baee83b64d2d25`
BLAKE2b-256	`97d82953280ea25b2d1b25e7952abbd45bcb8ee06c39b12de1dc60017f41374b`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai-0.1.0.dev4-cp310-abi3-win_arm64.whl:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai-0.1.0.dev4-cp310-abi3-win_arm64.whl
- Subject digest: a493e9597ee01d17cde5ebc1b5c5ba580370d470c6a6122de2cb1ee1d54efe87
- Sigstore transparency entry: 2238675364
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

File details

Details for the file onnx_genai-0.1.0.dev4-cp310-abi3-win_amd64.whl.

File metadata

Download URL: onnx_genai-0.1.0.dev4-cp310-abi3-win_amd64.whl
Upload date: Jul 24, 2026
Size: 5.0 MB
Tags: CPython 3.10+, Windows x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai-0.1.0.dev4-cp310-abi3-win_amd64.whl
Algorithm	Hash digest
SHA256	`47422edaafc96beb10500f713d7d71685719b87e27392f526530cd2b813af74b`
MD5	`e5f6eafc624473aff057a6cdad9bcb14`
BLAKE2b-256	`12f2d0e4940f1a5f0b0bf9ee63fd28185bb20f7f832e4cc9bc3e03afe3116c49`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai-0.1.0.dev4-cp310-abi3-win_amd64.whl:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai-0.1.0.dev4-cp310-abi3-win_amd64.whl
- Subject digest: 47422edaafc96beb10500f713d7d71685719b87e27392f526530cd2b813af74b
- Sigstore transparency entry: 2238675698
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

File details

Details for the file onnx_genai-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl.

File metadata

Download URL: onnx_genai-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl
Upload date: Jul 24, 2026
Size: 5.1 MB
Tags: CPython 3.10+, manylinux: glibc 2.28+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl
Algorithm	Hash digest
SHA256	`ae68efe404af27acea12eb27d49f8191cfd934fbaafa29ccd14af632ea19c35e`
MD5	`d66f95a5f438982e47a095028e1bfd20`
BLAKE2b-256	`207b634082a53fd5d3fb68739de5d14505aabe57f51bf0be55053cf2e7629748`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl
- Subject digest: ae68efe404af27acea12eb27d49f8191cfd934fbaafa29ccd14af632ea19c35e
- Sigstore transparency entry: 2238675041
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

File details

Details for the file onnx_genai-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

Download URL: onnx_genai-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl
Upload date: Jul 24, 2026
Size: 4.5 MB
Tags: CPython 3.10+, macOS 11.0+ ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl
Algorithm	Hash digest
SHA256	`b7c5bb65b84c77fffb07361422a8799addafbbe211890bef06f20fd0ea6ea63a`
MD5	`2179134e2fec3ee5acbed01ffc9fe2cc`
BLAKE2b-256	`e16ebdbe5ce4fbdcd890e132d9be15c6e267104734f2e99550033aef8291282d`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl
- Subject digest: b7c5bb65b84c77fffb07361422a8799addafbbe211890bef06f20fd0ea6ea63a
- Sigstore transparency entry: 2238674755
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

Release history Release notifications | RSS feed

Pre-release

0.1.0.dev6

Sep 10, 2026 5 files

This release

Pre-release

0.1.0.dev4 This release

Jul 24, 2026 5 files

Pre-release

0.1.0.dev3

Jul 21, 2026 5 files

Yanked

0.0.0

Jul 18, 2026 1 file

onnx-genai 0.1.0.dev4

onnx-genai

Wheels

Related packages

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License

Requires

Provides Extra

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

Release history Release notifications | RSS feed