Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs        # CPU, or Metal on Apple Silicon

NVIDIA CUDA wheels ship as GitHub release assets because they vary by CUDA toolkit level (cuda128, cuda129, cuda131) and GPU compute capability. See the Python SDK installation guide for the --find-links command.

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

This version

0.8.23

0.8.22

0.8.21

0.8.20

0.8.18

0.8.17

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

0.8.9

0.8.8

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.23-cp310-abi3-win_amd64.whl (20.2 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.23-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.8 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.23-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (22.2 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.23-cp310-abi3-macosx_11_0_arm64.whl (36.5 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.23-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.23-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.2 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.23-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 7806b1b67181f7a5ef1aed112c2b58e469bfeec2be8c3e6892eaf274aff14c74
MD5 2d6edd38cf7fd30642aff83d9cebc3b0
BLAKE2b-256 d4e4c54d35970da1f0b6f062ed2d0d7dafcc69dc9cf2e9edd5e8481ce821a7ab

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.23-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.23-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.23-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 aff6e0e5f44708332b81ab16df8f1279b597863320bea12ae6ca8d50cbaa6e08
MD5 ca4ce369f22405aff2cbaef59e8f9626
BLAKE2b-256 c83f7a246311683ae0caa521cbe3e243c9e96cdc4a2a19466497530b79a5b6a0

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.23-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.23-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.23-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 cf6d0848a731b26429767b3f80db7f835dc6f288b1fbf6c1fc46459d20a0d2a2
MD5 d375f10334b13dc5fa00da983fb07dd1
BLAKE2b-256 bec724209a51817c79689c0d02514ce907b37183e4db30f6578f2ad5b864aad3

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.23-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.23-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.23-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 bcc60ba8cb4663169a4813dafcf77580fa73882d55970e4df9ed92f3c009620d
MD5 4e2f2800c3eb5767999044b3eda7c9b3
BLAKE2b-256 16346cff68d51bc6306dd355de0d5b4033805e28ae7acf3318b8e0f140e2ca6a

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.23-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page