Pre-release

This release is a pre-release and may not be stable for production use.

onnx-genai-server

The onnx-genai command-line tool and OpenAI-compatible server, backed by ONNX Runtime.

pip install onnx-genai-server            # CPU (all platforms; MLX on macOS)
pip install onnx-genai-server[cuda]      # + CUDA 13 / cuDNN 9 (Windows/Linux)

ONNX Runtime is not bundled. The command loads libonnxruntime from the installed onnxruntime (CPU) or onnxruntime-gpu (CUDA) wheel at startup, so it uses whichever execution providers you installed.

Commands

onnx-genai serve --models-dir ./models       # OpenAI-compatible HTTP server
onnx-genai generate --model ./model "Hello"  # one-shot generation
onnx-genai run --model ./model               # interactive REPL
onnx-genai show ./model                       # resolved files + metadata
onnx-genai list --models-dir ./models         # list models
onnx-genai version                            # version + execution providers

generate, run, and show accept either a model directory or a config file inside it (a file resolves to its parent directory).

Runtime selection

Choose an execution provider at runtime with ONNX_GENAI_EP (e.g. cpu, cuda). CUDA requires the [cuda] extra (or a separately installed onnxruntime-gpu). On Apple Silicon, the onnxruntime-ep-mlx plugin is installed by default.

Python 3.11+ is required (the onnxruntime dependency ships no earlier wheels).

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

onnx_genai_server-0.1.0.dev4.tar.gz (3.3 MB view details)

Uploaded Jul 24, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

onnx_genai_server-0.1.0.dev4-cp310-abi3-win_arm64.whl (9.0 MB view details)

Uploaded Jul 24, 2026 CPython 3.10+Windows ARM64

onnx_genai_server-0.1.0.dev4-cp310-abi3-win_amd64.whl (9.7 MB view details)

Uploaded Jul 24, 2026 CPython 3.10+Windows x86-64

onnx_genai_server-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl (9.7 MB view details)

Uploaded Jul 24, 2026 CPython 3.10+manylinux: glibc 2.28+ x86-64

onnx_genai_server-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl (8.8 MB view details)

Uploaded Jul 24, 2026 CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file onnx_genai_server-0.1.0.dev4.tar.gz.

File metadata

Download URL: onnx_genai_server-0.1.0.dev4.tar.gz
Upload date: Jul 24, 2026
Size: 3.3 MB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai_server-0.1.0.dev4.tar.gz
Algorithm	Hash digest
SHA256	`c53d507f44fa9e614a4ac61b796dc2763f33eba92b10b66b0051e92f26c2f7ab`
MD5	`51e940a6d811c768dee03fb974132c6e`
BLAKE2b-256	`4d977f074b2f95a4b1f9e6f2338df8a537c4847f022440d0238a1cf15dd6a456`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai_server-0.1.0.dev4.tar.gz:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai_server-0.1.0.dev4.tar.gz
- Subject digest: c53d507f44fa9e614a4ac61b796dc2763f33eba92b10b66b0051e92f26c2f7ab
- Sigstore transparency entry: 2238558662
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

File details

Details for the file onnx_genai_server-0.1.0.dev4-cp310-abi3-win_arm64.whl.

File metadata

Download URL: onnx_genai_server-0.1.0.dev4-cp310-abi3-win_arm64.whl
Upload date: Jul 24, 2026
Size: 9.0 MB
Tags: CPython 3.10+, Windows ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai_server-0.1.0.dev4-cp310-abi3-win_arm64.whl
Algorithm	Hash digest
SHA256	`212fa4304d5e8df0a3712c66874123d1ef8d1bec0709b39059d3dd168210a3ef`
MD5	`c6b772ece187e9ef6a94d5d063aad4e7`
BLAKE2b-256	`0a36817380e1504d403975f13280932acdcd1abce874e6d707a43d3a2c12c0c9`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai_server-0.1.0.dev4-cp310-abi3-win_arm64.whl:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai_server-0.1.0.dev4-cp310-abi3-win_arm64.whl
- Subject digest: 212fa4304d5e8df0a3712c66874123d1ef8d1bec0709b39059d3dd168210a3ef
- Sigstore transparency entry: 2238675934
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

File details

Details for the file onnx_genai_server-0.1.0.dev4-cp310-abi3-win_amd64.whl.

File metadata

Download URL: onnx_genai_server-0.1.0.dev4-cp310-abi3-win_amd64.whl
Upload date: Jul 24, 2026
Size: 9.7 MB
Tags: CPython 3.10+, Windows x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai_server-0.1.0.dev4-cp310-abi3-win_amd64.whl
Algorithm	Hash digest
SHA256	`313be539ad76b9692da8d5d94ddaaddb16eb52c05beb9f6dfb5a832224241c95`
MD5	`46dbbcc59bdd7e7e17281699b3e114f1`
BLAKE2b-256	`477f6a23e9e660c236db3af3f46578b423e967b3a5dc8f142fe34f161c29eeb8`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai_server-0.1.0.dev4-cp310-abi3-win_amd64.whl:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai_server-0.1.0.dev4-cp310-abi3-win_amd64.whl
- Subject digest: 313be539ad76b9692da8d5d94ddaaddb16eb52c05beb9f6dfb5a832224241c95
- Sigstore transparency entry: 2238676348
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

File details

Details for the file onnx_genai_server-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl.

File metadata

Download URL: onnx_genai_server-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl
Upload date: Jul 24, 2026
Size: 9.7 MB
Tags: CPython 3.10+, manylinux: glibc 2.28+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai_server-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl
Algorithm	Hash digest
SHA256	`be6eef4e66c9e61df4d3ac6dc9ccddf4ddf2e7dccfa358286611fdca6124e00e`
MD5	`d465a3a448b5ae33123a8c3f9044c1f3`
BLAKE2b-256	`ad4d7d20ea9a9ccae65a9f34b527d680a195da8f47b491080036945dfc4b0f80`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai_server-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai_server-0.1.0.dev4-cp310-abi3-manylinux_2_28_x86_64.whl
- Subject digest: be6eef4e66c9e61df4d3ac6dc9ccddf4ddf2e7dccfa358286611fdca6124e00e
- Sigstore transparency entry: 2238675543
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

File details

Details for the file onnx_genai_server-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

Download URL: onnx_genai_server-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl
Upload date: Jul 24, 2026
Size: 8.8 MB
Tags: CPython 3.10+, macOS 11.0+ ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for onnx_genai_server-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl
Algorithm	Hash digest
SHA256	`81b9efe2fdf4ada32633bdccd850b16a3ce46e7206360cac7b00a44ccd0c017a`
MD5	`ac192236453c56aded5d81f10ae03224`
BLAKE2b-256	`4f01c76ebb46eb7bece75dbc052636ebbeb1bf5e6d2187efab84a2e65d63c30c`

See more details on using hashes here.

Provenance

The following attestation bundles were made for onnx_genai_server-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: publish.yml on justinchuby/onnx-genai

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: onnx_genai_server-0.1.0.dev4-cp310-abi3-macosx_11_0_arm64.whl
- Subject digest: 81b9efe2fdf4ada32633bdccd850b16a3ce46e7206360cac7b00a44ccd0c017a
- Sigstore transparency entry: 2238674494
- Sigstore integration time: Jul 24, 2026
Source repository:
- Permalink: justinchuby/onnx-genai@43048422dd33eb11e3e0c01e09d43365b3fab956
- Branch / Tag: refs/heads/main
- Owner: https://github.com/justinchuby
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@43048422dd33eb11e3e0c01e09d43365b3fab956
- Trigger Event: workflow_dispatch

Release history Release notifications | RSS feed

This release

Pre-release

0.1.0.dev4 This release

Jul 24, 2026 5 files

Pre-release

0.1.0.dev3

Jul 21, 2026 5 files

onnx-genai-server 0.1.0.dev4

onnx-genai-server

Commands

Runtime selection

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License

Requires

Provides Extra

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

Release history Release notifications | RSS feed