Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs        # CPU, or Metal on Apple Silicon

NVIDIA CUDA wheels ship as GitHub release assets because they vary by CUDA toolkit level (cuda128, cuda129, cuda131) and GPU compute capability. See the Python SDK installation guide for the --find-links command.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

0.8.23

0.8.22

0.8.21

0.8.20

0.8.18

0.8.17

This version

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

0.8.9

0.8.8

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.16-cp310-abi3-win_amd64.whl (20.0 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.16-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.16-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (22.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.16-cp310-abi3-macosx_11_0_arm64.whl (36.3 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.16-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.16-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.0 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.16-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 bb70eebf17614bf8b37e37dc238d215a816b23c11a335f307fc1f441d41d0fa9
MD5 f27beb420ee05970ca23cf3513333aed
BLAKE2b-256 a0e1e1c5256d74f1e0d69ea5d3e6273e10898d47135348960c30399ad9ca4914

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.16-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.16-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.16-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a63dd609f69baa4bc99e0308188ae1ce0d49995ea264f5049361429e3086ee14
MD5 57738c5f3e83d7d6af9360967e2e98ba
BLAKE2b-256 c0abdde8721ad7288760249dfcca33586e16f2dc8ca1c1ecccb3d3030d64b556

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.16-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.16-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.16-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ecfc9898bb5ac6810a14e26d5aa7d663b75a20521cc93d9f36377a32d61f3deb
MD5 f685285025c09f7675841474126a967d
BLAKE2b-256 cd7ac2a84dbeca52f90cbc7e52ee750ce5e4ba5b7c839a5ab105dae75f766d28

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.16-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.16-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.16-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0a41c64674d009fea6798f43abca7205f22ee6b2ce3e25464d17357cbf4d039c
MD5 1eef9de4fc525f55ad3645c87829f9a7
BLAKE2b-256 fcd58784986c012e82068d2693475e313717b9123bc402f840af2ecb2ee81ada

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.16-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page