ONNX Runtime generate() API

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI
Meta

Project description

ONNX Runtime generate() API

Run SLMs/LLMs and multi modal models on-device and in the cloud with ONNX Runtime.

Model architectures supported so far (and more coming soon): Gemma, Llama, Mistral, Phi (language and vision).

For more details, see: docs https://onnxruntime.ai/docs/genai and repo: https://github.com/microsoft/onnxruntime-genai

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI
Meta

Release history Release notifications | RSS feed

0.5.1

This version

0.5.0

0.4.0

0.3.0

0.3.0rc2 pre-release

0.3.0rc1 pre-release

0.2.0

0.2.0rc7 pre-release

0.2.0rc6 pre-release

0.2.0rc5 pre-release

0.2.0rc4 pre-release

0.2.0rc3 pre-release

0.2.0rc2 pre-release

0.2.0rc1 pre-release

0.1.0

0.1.0rc4 pre-release

0.1.0rc3 pre-release

0.1.0rc2 pre-release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

onnxruntime_genai-0.5.0-cp312-cp312-win_arm64.whl (728.4 kB view details)

Uploaded CPython 3.12 Windows ARM64

onnxruntime_genai-0.5.0-cp312-cp312-win_amd64.whl (754.3 kB view details)

Uploaded CPython 3.12 Windows x86-64

onnxruntime_genai-0.5.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.3 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.0-cp312-cp312-macosx_12_0_x86_64.whl (845.3 kB view details)

Uploaded CPython 3.12 macOS 12.0+ x86-64

onnxruntime_genai-0.5.0-cp312-cp312-macosx_12_0_arm64.whl (747.3 kB view details)

Uploaded CPython 3.12 macOS 12.0+ ARM64

onnxruntime_genai-0.5.0-cp311-cp311-win_arm64.whl (728.3 kB view details)

Uploaded CPython 3.11 Windows ARM64

onnxruntime_genai-0.5.0-cp311-cp311-win_amd64.whl (753.9 kB view details)

Uploaded CPython 3.11 Windows x86-64

onnxruntime_genai-0.5.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.3 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.0-cp311-cp311-macosx_12_0_x86_64.whl (843.0 kB view details)

Uploaded CPython 3.11 macOS 12.0+ x86-64

onnxruntime_genai-0.5.0-cp311-cp311-macosx_12_0_arm64.whl (746.1 kB view details)

Uploaded CPython 3.11 macOS 12.0+ ARM64

onnxruntime_genai-0.5.0-cp310-cp310-win_amd64.whl (753.7 kB view details)

Uploaded CPython 3.10 Windows x86-64

onnxruntime_genai-0.5.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.3 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.0-cp310-cp310-macosx_12_0_x86_64.whl (842.9 kB view details)

Uploaded CPython 3.10 macOS 12.0+ x86-64

onnxruntime_genai-0.5.0-cp310-cp310-macosx_12_0_arm64.whl (746.1 kB view details)

Uploaded CPython 3.10 macOS 12.0+ ARM64

File details

Details for the file onnxruntime_genai-0.5.0-cp312-cp312-win_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp312-cp312-win_arm64.whl
Algorithm Hash digest
SHA256 d6f163051663121145db7959798c66323bb6df25d979632720d259c9e139fe4f
MD5 b6cb0470fa8ddebd958efd7a1e8d4161
BLAKE2b-256 22b074a54df85342eb95b063f87c2cb61936997a73e9fbe955584635d9bad612

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 654a375379c11a69b943d6d15486ddde9736ce107a15c5ae73b813c4b5753533
MD5 a81e7dd482da3817b2feea1a10024311
BLAKE2b-256 9710c246ebd158cbad4c04f14d09f2269d6856e1224b0ab7783aab491c36f0f1

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 827f1195690f059fda788d7d7e42d63b5d626782a7edc69d66385ecca1a5db9a
MD5 dc16abe4a2d21d4e55caa0c8c690cdb8
BLAKE2b-256 b3bb0693a00c30224c2a28987f7050e15efccf6a36c5c314668c686fec2341ac

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp312-cp312-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp312-cp312-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 171df9e2d7b673ebdc74694a1e44ee95f27efdf2d5d28f336c87280e5ed8f9fe
MD5 3a8bb55100adbbfc37ba291fce343fc9
BLAKE2b-256 df6acc915ee882579797fcd91fc003fa0f392ce4a2b66006a5db7564d1795584

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp312-cp312-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp312-cp312-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 010ffe69d1d91d295b678f8dc3996c6f4b366c8e08c8f4c82209a61ae8c773a9
MD5 d36b2e7353200ed938857e4e85a93713
BLAKE2b-256 faae98eae953410d523e8cec8853879f07a94202204fc8265f1d579589012cf6

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp311-cp311-win_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp311-cp311-win_arm64.whl
Algorithm Hash digest
SHA256 4392bb88eaa1b56cfeb426f350828c41c67ff98c239cd7e35aff1da1f84e31fb
MD5 520443e5b700f481b2de4dfbfd16e360
BLAKE2b-256 8be6781b1b0e5f90eaf14c4113f6e52f440d956e79318802548c045fbd20c2d8

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 cb6f80aefd71b320957f8fa42a62a76702087a5c891f22393d657fef9237a157
MD5 16c60ac58a5f1dc733e19858434b4099
BLAKE2b-256 8435cee550b2c50f582dcc78010f117eaf94b4bf01f13de1fa75b4cd3426dfa6

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 352d4523b65d40c21223ae3540b1fe981853d532143f3a58135fcaf7d5761334
MD5 83e88f41c0a706323734edd0b131873c
BLAKE2b-256 6a652d0bfa2e3bef7b6c9d02c008cc2c6d63857ffbe3722675e3eb058c3fc0de

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp311-cp311-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp311-cp311-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 56362d83f49ff176d3169660887ecc4a7c849296465c2fa8ff705aa5152dbe3c
MD5 c73a356f4fb7871d8c029ad838f716f6
BLAKE2b-256 a555a54a5667fc920c1d1d44ce75ee367ea9240c342c40267f351c10ad031325

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp311-cp311-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp311-cp311-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 089f59c40f3785ff99eb07bcd0ebc7b5b37e422aa9f0ed2774874a0cf8854ad5
MD5 15c215b62fb40d492d2dab502e190b61
BLAKE2b-256 fbcfd043f36d215be8d38a6ee0d2172506295fd6217afd3693091d9c95b3c8e9

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 a45870372d1ca8290737199f0845c14f2053dd26c8f0f3df39c132cfff4f8e60
MD5 dbf67368ca2798d255de5e0475d15f03
BLAKE2b-256 5bddf5ae45f44a04fed29397768a91c03143fec2b4630e42a4628a5b1aa05f2b

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 eefc63740d3a10ddf9eed767e4f23aac8da5e765000247dbada225ebc280cf3a
MD5 dbc54c37f8a1778b1bd0e1c8c11b3e1d
BLAKE2b-256 b251b3f8e0b72fde5fb7c001029acb3fadc15bc11277457def8d791ef3c2bcae

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp310-cp310-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp310-cp310-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 ce64f46f8c29fcc252311fe12ce42266a979e873d9281ffc2640a9b3ecd71464
MD5 5214ceaaeb00e41f69328945b038f3f3
BLAKE2b-256 864e9c25075a9f74fbe9b7bc6d78941036559c0752b06a4ff124d8c8a1668c84

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.0-cp310-cp310-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.0-cp310-cp310-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 04fb1773eb1c24825c15776c444f1a2be4038d42410f0d8e2a7bab5ef0bf58de
MD5 0637a3c2630078840dcad01c6a950ff5
BLAKE2b-256 8bb40dd1fd22808bcdce8509471c7c7a5ca62130dc6b3e5d2345c62b2503a53a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page