Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs        # CPU, or Metal on Apple Silicon

NVIDIA CUDA wheels ship as GitHub release assets because they vary by CUDA toolkit lane (cuda128, cuda129, cuda130, cuda131, cuda132, cuda133) and GPU compute capability. See the Python SDK installation guide for the --find-links command.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

This version

0.9.0

0.8.23

0.8.22

0.8.21

0.8.20

0.8.18

0.8.17

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

0.8.9

0.8.8

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.9.0-cp310-abi3-win_amd64.whl (20.7 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.9.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23.4 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.9.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (22.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.9.0-cp310-abi3-macosx_11_0_arm64.whl (36.6 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.9.0-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.9.0-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.7 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.9.0-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 8fdfa34ac175ec560f22fcde8aec61aaff1b680a22c3755203f875f68245e06e
MD5 7c980c74582ccc4dff27a9b0ebd6fe45
BLAKE2b-256 b9c1a75d14404faca5613f7b6d4f24b3cb3c29ac9995345db3d3b07b150e7b83

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.9.0-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.9.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.9.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8514487856acd10318114791e196cf3a52d9acdccabf6c153437fa586f058a76
MD5 78597ff14f797f90f5f88a65290c1a6b
BLAKE2b-256 85a6f178d1e01ddbf1305f629fb396d2b1797e3cee746516fe51d8412aa082a8

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.9.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.9.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.9.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 923050665d76c1c7f4dc264c551411dada06768d8d1fc1394b01888d86fd3e0a
MD5 8a676772d5c226331301e04c94244bfa
BLAKE2b-256 d8b68970251ae8ff9dbb052eec95007cc27de9f0d84f78e8d18da00b2d30b862

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.9.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.9.0-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.9.0-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 aa0c5e20eb7538b5c8fd4403a02819daec9bb60e74955180e8a1397b69d9f6b1
MD5 1943719b2b2f3c1479efddc5a1a83606
BLAKE2b-256 8a232dbc60ea14e16d23a5cd9d076d3327710f9f5fdfb9ab18f196706f8a8854

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.9.0-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page