ONNX Runtime generate() API

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI
Meta

Project description

ONNX Runtime generate() API

Run SLMs/LLMs and multi modal models on-device and in the cloud with ONNX Runtime.

Model architectures supported so far (and more coming soon): Gemma, Llama, Mistral, Phi (language and vision).

For more details, see: docs https://onnxruntime.ai/docs/genai and repo: https://github.com/microsoft/onnxruntime-genai

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI
Meta

Release history Release notifications | RSS feed

0.5.1

0.5.0

This version

0.4.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

onnxruntime_genai_cuda-0.4.0-cp312-cp312-win_amd64.whl (14.5 MB view details)

Uploaded CPython 3.12 Windows x86-64

onnxruntime_genai_cuda-0.4.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.3 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai_cuda-0.4.0-cp311-cp311-win_amd64.whl (14.5 MB view details)

Uploaded CPython 3.11 Windows x86-64

onnxruntime_genai_cuda-0.4.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.3 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai_cuda-0.4.0-cp310-cp310-win_amd64.whl (14.5 MB view details)

Uploaded CPython 3.10 Windows x86-64

onnxruntime_genai_cuda-0.4.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.3 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai_cuda-0.4.0-cp39-cp39-win_amd64.whl (14.5 MB view details)

Uploaded CPython 3.9 Windows x86-64

onnxruntime_genai_cuda-0.4.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.3 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai_cuda-0.4.0-cp38-cp38-win_amd64.whl (14.5 MB view details)

Uploaded CPython 3.8 Windows x86-64

onnxruntime_genai_cuda-0.4.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (15.3 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 3f97668e1714714edbb48f46ef6cdcfea4adca5b7c1ec7984b9f147ce8b2b434
MD5 ea2ae778bc160dd96fd00c068df7c2ad
BLAKE2b-256 0d5e95218d85b7b7566b071ccab6e728ea2dafd24c5987cd85dc7ea381e3df6d

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 1cbe67f99821c6928ee379e10892aebc4527a249b028f2933ca18e52bd3c1452
MD5 2d66a5cae00df934f75d47384ead38e4
BLAKE2b-256 74429025fb9cc0706256fda67bf7d578677ae6bb24137c7d59786633004b2f41

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 98c873e92328d07f13b5326d6d45a9d4172a56eb36317f19d03dd4a305eeeab7
MD5 eda5216c2264d262b36db53ba1a9b3a2
BLAKE2b-256 9b4d89db5ed3447208f1f4f40d22cb6ed0f958c42eb9f8c8d9e71483499d9006

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 2e8ff90d1d4fc8eb1ebfeff389c3635dc6fca23c8f37529e2be1939eb84fb2eb
MD5 8f72d0ced3e8f0d54ec3af2d9a36c48c
BLAKE2b-256 53c4745d8895aeb910ef8310ee79f017df7555e66b2dc6515c5bf0f7f36c8aca

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 a04b6e6a074be16ffbe83555400e3578e40131d95db7931cbb0d379e88fc7a48
MD5 6bd24cf1f616437a4499251b17190ad3
BLAKE2b-256 b187751b0d9e7b908b1749cc4d018df9a8b3ca2c9258b37c139523bbfb0c641a

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 8cdbbaac423ce2a8c9c61872348a79e9c696e19b11a85b3c75cab71e3448c538
MD5 df816b8d8c2fd6bd7fe5aeefa8259b12
BLAKE2b-256 6b2a64433015fad2f584d65dd4ee87db74e77aaffd1ba8540e1df5bfde3a5dfc

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp39-cp39-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 c3fb1e7543f80a438aea2e07129f3586ad2b789e64a329d819735801fbcc620e
MD5 749c4d7a6fba040f4ca97a4ed8d9e607
BLAKE2b-256 cc78e10ae93e80287656c189ec66ca8061f48126bf661a2915c39e4f5406312f

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 f18ebcf4cf82785471d564b7b164cb3488cca304122603493892bd3282e5e6ff
MD5 4a09d3bb8a3ba492bfcef1f958f35e6e
BLAKE2b-256 feadd3b4fc569feb2b9261ff2a8edf36dce9fe4805f120b1de398c24d1106feb

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp38-cp38-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 7cb359aea703a5edad4355eb6d58299413b183dd229bf7971cac9af5592d0d2e
MD5 e16a35679e511a357a4602556e7483c2
BLAKE2b-256 2c56cdf39827a38215fb3b2f72b7a785f5c71e8c27ed83819b8d26d228e81856

See more details on using hashes here.

File details

Details for the file onnxruntime_genai_cuda-0.4.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai_cuda-0.4.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 73cb3626a9f8fe31cdfc36190b114faf2aaeff26edafbe3ef5a143f3fcb4cac0
MD5 f15566f328e47dbf1868c4acc673058d
BLAKE2b-256 0663f610f636bdcd9629cf8ddc7f806190840eee1ef8bf910f7ba71da2e961de

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page