ONNX Runtime generate() API

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI
Meta

Project description

ONNX Runtime generate() API

Run SLMs/LLMs and multi modal models on-device and in the cloud with ONNX Runtime.

Model architectures supported so far (and more coming soon): Gemma, Llama, Mistral, Phi (language and vision).

For more details, see: docs https://onnxruntime.ai/docs/genai and repo: https://github.com/microsoft/onnxruntime-genai

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI
Meta

Release history Release notifications | RSS feed

This version

0.5.1

0.5.0

0.4.0

0.3.0

0.3.0rc2 pre-release

0.3.0rc1 pre-release

0.2.0

0.2.0rc7 pre-release

0.2.0rc6 pre-release

0.2.0rc5 pre-release

0.2.0rc4 pre-release

0.2.0rc3 pre-release

0.2.0rc2 pre-release

0.2.0rc1 pre-release

0.1.0

0.1.0rc4 pre-release

0.1.0rc3 pre-release

0.1.0rc2 pre-release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

onnxruntime_genai-0.5.1-cp312-cp312-win_arm64.whl (751.2 kB view details)

Uploaded CPython 3.12 Windows ARM64

onnxruntime_genai-0.5.1-cp312-cp312-win_amd64.whl (777.0 kB view details)

Uploaded CPython 3.12 Windows x86-64

onnxruntime_genai-0.5.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.1-cp312-cp312-macosx_12_0_x86_64.whl (871.4 kB view details)

Uploaded CPython 3.12 macOS 12.0+ x86-64

onnxruntime_genai-0.5.1-cp312-cp312-macosx_12_0_arm64.whl (771.1 kB view details)

Uploaded CPython 3.12 macOS 12.0+ ARM64

onnxruntime_genai-0.5.1-cp311-cp311-win_arm64.whl (751.5 kB view details)

Uploaded CPython 3.11 Windows ARM64

onnxruntime_genai-0.5.1-cp311-cp311-win_amd64.whl (776.3 kB view details)

Uploaded CPython 3.11 Windows x86-64

onnxruntime_genai-0.5.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.1-cp311-cp311-macosx_12_0_x86_64.whl (869.0 kB view details)

Uploaded CPython 3.11 macOS 12.0+ x86-64

onnxruntime_genai-0.5.1-cp311-cp311-macosx_12_0_arm64.whl (769.9 kB view details)

Uploaded CPython 3.11 macOS 12.0+ ARM64

onnxruntime_genai-0.5.1-cp310-cp310-win_amd64.whl (776.3 kB view details)

Uploaded CPython 3.10 Windows x86-64

onnxruntime_genai-0.5.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.1-cp310-cp310-macosx_12_0_x86_64.whl (869.1 kB view details)

Uploaded CPython 3.10 macOS 12.0+ x86-64

onnxruntime_genai-0.5.1-cp310-cp310-macosx_12_0_arm64.whl (770.0 kB view details)

Uploaded CPython 3.10 macOS 12.0+ ARM64

File details

Details for the file onnxruntime_genai-0.5.1-cp312-cp312-win_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp312-cp312-win_arm64.whl
Algorithm Hash digest
SHA256 8b77790d105b0f80b0548f5125600f6e22e711f7a9e0054ed96f43ff8674d1ca
MD5 6360eadadc5cf26405fd8366e2ff1165
BLAKE2b-256 952eef708e666f5c636d0100e120a6d0a6e5faec1ff2872ead2c7a4be8db102f

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 be10c977e28355f316439416bd86ea007f51c728fa9653afff2758f41c9abc8c
MD5 c5f7cc48c170e862236a93ea47e64ec7
BLAKE2b-256 a31184a65930fc9e3c38751bb402abc47b473ab55dc0352610b362ccc073fb1c

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 30487396cf5d2c806a9993e2096c6eaab6bc9d55869af48a11e8c783fac65b41
MD5 fde2643dd671d1c2ac4ab56a95f1b1c2
BLAKE2b-256 dd3a5cc18cfc54bd074e4f15a28d8c0da8595f18f1149b27b295b3738ff7b9e4

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp312-cp312-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp312-cp312-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 5182e0154b7112a6842e0171802e9dd750183be3b3b718c00bf7ef303dd47fb9
MD5 26885e27e28a8d807ec845b39f590517
BLAKE2b-256 a18fffac04fc3f8b554c0f7df860845ecfc6dfdfea9d4f1a933ad76e601b4650

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp312-cp312-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp312-cp312-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 7ecc73ebba89e6aa0d11bb73b6a3c8dbf26c4e9443f39368d3d7279697d12a2f
MD5 9aca1baf6e3edd9d885b18c1b534036a
BLAKE2b-256 8d0814216dfd1458f65d42fa67f514fc7a6587b0dfb25949c911ba56d6aadf11

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp311-cp311-win_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp311-cp311-win_arm64.whl
Algorithm Hash digest
SHA256 e8d0e5ff3d07bdce91eba57ff38baa08c3ad53045f5191c82c88eae5b4119b41
MD5 8c15ebabbf209f9ab8377e9e70adfbcf
BLAKE2b-256 b4112ba0398223955dac82d84417b492a9ae0be23e05569d444c6b7a07c97c00

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 a4d6e349f5fc89aa0524b0cd722fe381dafe2e8e6f3128540cf6ce71922a73ef
MD5 36617621167548678b018895a37a706b
BLAKE2b-256 d9d5d361760378403f5b71a9e487a20ef550dbe7874d7a91472ed9e82c3a2c41

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 4b399f4122371051a5ed6dea529e4ecf5697e27fe49bb0cc0714c40f81581af7
MD5 2be57554bc40801085263a7e0301e154
BLAKE2b-256 1e6aac5391cecf7729247391f2e76cbb095733493e4fb8667e1cabd48f2a87fc

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp311-cp311-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp311-cp311-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 8325350e2faa1a604b18fc77122fdb0115fc07bfc7544b08517d4a2451245b01
MD5 9ee517f0f577cb7277a9c382dabfc376
BLAKE2b-256 4d187078d1e567b363210f8fea16318ac846db0fa7c391ecc4b7cda90e639c32

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp311-cp311-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp311-cp311-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 f4133dabc14fdb6db89274817c5eb470b2c793a902d2cc41a99f3f48a0efe913
MD5 d790bd86ef574665d4a357f8c0106c3a
BLAKE2b-256 9ff6bf116ca3206954c9e462f44ca614351aa3c025edfcb426a86718da832393

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 92f753787b3c8e98e2039d7cc233aeebbdd2dbfaa6b5fe1dc4bfb2e984ea0430
MD5 e69cc0ad315579f472cd475eb033a954
BLAKE2b-256 4aa547dfb20e54982c16577b282f777e48e510b4da5738e13b38d9238e22f34c

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 9dbe45488608a81adf5f01d7ab1384c75833784167787d9aa8992290465c878b
MD5 a1603b064c900ace7a1fd1a7b58a6cd4
BLAKE2b-256 be7dff40bbae602bc667fe60ad739dca0a1759d14323b6ff285b6e10a0429d41

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp310-cp310-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp310-cp310-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 5f617899782d779fc661d8f53f780489d00a53ddaae66ff097e12766e55cbdfd
MD5 e5e7de935c371306bfe13d1b48dd6fab
BLAKE2b-256 ef74d69cb6c01cc5b5cefef3b76f7e8b574e6ec0acde8196bb6ce95a5f3fcf0b

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.1-cp310-cp310-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.1-cp310-cp310-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 db9e3e71f487846c146930366d8bd9d335b2a3163d4725e826693c95bff8e44d
MD5 41544c51731838f539de57262bc477ca
BLAKE2b-256 e7225c77f250ec94b29c5d9b1ca02c89ab417588a42fac34239bf61bce4cc9e6

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page