Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs        # CPU, or Metal on Apple Silicon

NVIDIA CUDA wheels ship as GitHub release assets because they vary by CUDA toolkit level (cuda128, cuda129, cuda131) and GPU compute capability. See the Python SDK installation guide for the --find-links command.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

0.8.23

0.8.22

0.8.21

0.8.20

0.8.18

0.8.17

0.8.16

0.8.15

0.8.14

This version

0.8.13

0.8.12

0.8.9

0.8.8

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.13-cp310-abi3-win_amd64.whl (20.0 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.13-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.13-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (22.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.13-cp310-abi3-macosx_11_0_arm64.whl (36.3 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.13-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.13-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.0 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.13-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 8ef73d0ef4cc2faabef8571810c729d4796223170839518ab78bcf2897fcc878
MD5 c218d8b58d48057c36bd7907bf0e02d8
BLAKE2b-256 a46cb7385bdd8a151e9da71223d4712c7efbbb384e858e97063f94d5bd965c4f

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.13-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.13-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.13-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f782050f5dc459dacf080687ee66d8041ceab92f3670d81b76e1b95518359831
MD5 ac65dc6a83c8f22549f1f9d312d18106
BLAKE2b-256 cef3c93832dab596f58d4ae915e260d3b493453ae5beb5371957fb42368cf296

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.13-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.13-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.13-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ee07193d2f90aef6c0353bdcec5d0eb5e93e6ea7b9ab8ab108c0021bc97d9061
MD5 49b719326f6da33fea2712dddfe109b2
BLAKE2b-256 6c6176f277d3d5e11ac1871d019cc6a59dd469404f7a157a2d8e40578da0a34b

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.13-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.13-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.13-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a25b4f4a214505d70380c7ecc5ccfbcbcd502d317d9916c3beb645a44d223c9a
MD5 154d8404d603d64de0b07d3f34fa5360
BLAKE2b-256 fbb76995034b4192714575c78f8cca46c7f50269f93ff05837560f82794b54c6

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.13-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page