Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs        # CPU, or Metal on Apple Silicon

NVIDIA CUDA wheels ship as GitHub release assets because they vary by CUDA toolkit level (cuda128, cuda129, cuda131) and GPU compute capability. See the Python SDK installation guide for the --find-links command.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

0.8.23

0.8.22

0.8.21

0.8.20

0.8.18

This version

0.8.17

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

0.8.9

0.8.8

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.17-cp310-abi3-win_amd64.whl (20.0 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.17-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.17-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (22.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.17-cp310-abi3-macosx_11_0_arm64.whl (36.3 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.17-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.17-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.0 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.17-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 b2ab61ef59c060ef26247bffd72d4b3b24f61ce45cd448340b7c06cdc17c6521
MD5 532f272c90a2916e667df5f17bd49c67
BLAKE2b-256 a1130b341d8d2acddd379adee9edee9aa511c79203060801e80c0d24fbbc6ec7

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.17-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.17-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.17-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 aaa5c844a9b5aefc557805d93de338a89f9ba4d5941e84965724a06e8fa5e48a
MD5 a8c1ea5d367cd32372264e51f48b846f
BLAKE2b-256 f0254b77bb93ff40b0ec704a173d9552399a44514d58d4880b53ebcce9277f5f

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.17-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.17-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.17-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 dc42f6ac96138e8f47a8dc30e1a0c42d0c6c1a6b4944a3256a2735eb165b4dbf
MD5 35a8d791302fa5181418e56b8f009bcf
BLAKE2b-256 1fa1088771b55a147283409f1e3cbb3b12e411c45ebbffc3f4c5de7240ec159f

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.17-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.17-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.17-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8ea66f94ccd6538293cca91d52b8f54c033945d2a832f9a69a23b3b3c437d3df
MD5 ddf26a89400b24f4d450a441da66e3b8
BLAKE2b-256 d059d5013c3d3ddfef0579fd12a164fa6399bff1dd7561932d6f26e579b8776d

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.17-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page