Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs        # CPU, or Metal on Apple Silicon

NVIDIA CUDA wheels ship as GitHub release assets because they vary by CUDA toolkit level (cuda128, cuda129, cuda131) and GPU compute capability. See the Python SDK installation guide for the --find-links command.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

0.8.23

0.8.22

0.8.21

0.8.20

This version

0.8.18

0.8.17

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

0.8.9

0.8.8

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.18-cp310-abi3-win_amd64.whl (20.0 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.18-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.18-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (22.1 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.18-cp310-abi3-macosx_11_0_arm64.whl (36.3 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.18-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.18-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.0 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.18-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 2ead3cd57065154bdac426b40dd7e36c9eccc552f692d0dc2d2a20c84d72c778
MD5 6e751dcbf1c76b6cf783cabb8cf80683
BLAKE2b-256 eaeeb811d3318c10bb89e01b4e4cfc09c2a138a6d152b4ff0e53f1117cbf15c5

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.18-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.18-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.18-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0da543b3f173b0b92815a3c91ad83fb48238fa5bec66c00c88ec96aaf49a677a
MD5 2ea9f0f11b16c8f638340fbe0efcd0d6
BLAKE2b-256 5e95b359ca86eadf6fb007f7b32bd07ebccdcbeb0c024d66efbded3100072225

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.18-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.18-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.18-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 af9b231d4614b2d77ebab4c5e12dd15ca2ca7bcce5b9ab1233ab512b5d8bb841
MD5 c1b42da2b210c21983c0a4c426f81446
BLAKE2b-256 bc20793134ce99699f081cbf40c49398a2123883835d4745dfbb31f35d03d4af

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.18-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.18-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.18-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 66eb6f6421f2851bf634caabc82bf6930e9e2142c20d09e9bdde696b21a8f8c4
MD5 b1f0ca2fdd56d8d0c7c9a58ed07e84ff
BLAKE2b-256 5d6df2cea274a078d437b2efb99cab6d1603439de7691103d385a9af93e3407d

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.18-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page