Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs-cuda   # NVIDIA GPUs
pip install mistralrs-metal  # Apple Silicon
pip install mistralrs        # CPU only

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

0.8.23

0.8.22

0.8.21

0.8.20

0.8.18

0.8.17

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

0.8.9

0.8.8

This version

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.6-cp310-abi3-win_amd64.whl (20.1 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.6-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23.1 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.6-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (21.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.6-cp310-abi3-macosx_11_0_arm64.whl (36.3 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.6-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.6-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.1 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.6-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 f016d2f00fbcdb3e0740f2f2e3381a83cc3cdf108b33a98a2cd3977d492224d0
MD5 dbbc275ee58b97aebd3ef908fd7d007a
BLAKE2b-256 220d8084c38f214445d1673f39d0e5a231372df8a66cff0e7fea4a3c6b329a83

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.6-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.6-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.6-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9faa368b05e176f9b2f3396a61ac512f080bb7214a72d6ed99bbfbeabdbd89e1
MD5 1dcd71be01202af4269262fb527aa02c
BLAKE2b-256 85e52a642792f8416cdc97eb6411b1e410276a78fd44bd4d993e9078065c4218

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.6-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.6-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.6-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 2a028ded9ae1336092e923df500bc62747c8049e40bb8fb66720f1cf59359486
MD5 5c69ce0aa24a1af92976e4f8a202d12e
BLAKE2b-256 3a647d767493a9a73dad0533f4e9c156841ee7d68491f17b1679c9a8428664a7

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.6-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.6-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.6-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 89d8265acb6bbba0dbf0eb4af8d11142b1e234a0dc7e890ce175afc726eae0a1
MD5 07590dd4a31a8280f17bbc66078ff9e4
BLAKE2b-256 91c37da20e9160ca27b3b706fe8f664044275d12915a8cc7bc7351fe5a41aa25

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.6-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page