Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs        # CPU, or Metal on Apple Silicon

NVIDIA CUDA wheels ship as GitHub release assets because they vary by CUDA toolkit level (cuda128, cuda129, cuda131) and GPU compute capability. See the Python SDK installation guide for the --find-links command.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

0.8.23

0.8.22

0.8.21

This version

0.8.20

0.8.18

0.8.17

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

0.8.9

0.8.8

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.20-cp310-abi3-win_amd64.whl (20.0 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.20-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.20-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (22.1 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.20-cp310-abi3-macosx_11_0_arm64.whl (36.3 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.20-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.20-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.0 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.20-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 1e1becb4bba219a16a4e82cff032ea537d6736fb6b3a9ed48d23f0cfe05a7658
MD5 16a221eda52ea760de331f9bd5df266f
BLAKE2b-256 9937498e5fa78c06959eb9ea6e84df96ef111b94a66ef02245f54658eb8e2c49

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.20-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.20-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.20-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 04a855da0982a17a2e265d0f5ee91cf1a1b80c251f3a7886db33b6b1e854ed22
MD5 4deabb705677800982d5f75d3ccd46fd
BLAKE2b-256 6268fb12c179f606ce1fd7a29f45397c520339523118ccfeb7ffaa617c8f9f13

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.20-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.20-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.20-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 4b69757f860ee209a4656814a798f2c5c0306a846f41a48ff43fef1275bf209b
MD5 5e65c8f960aeec5daa0bff73b87cc066
BLAKE2b-256 ac49198c4bb240c624ae4c049d6d5595badf8738d5a223db9021219c44515fb4

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.20-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.20-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.20-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 96eb06d37a611e4096ef5421fa8298f41138319fa39d4d4bda55c4343f07c183
MD5 925d458fa87997a9a69c46d87d3dbb6d
BLAKE2b-256 06406e65781ce57a43bc731b7758ab00d6a5dd2081cc98d882c39f1318f367c7

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.20-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page