Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs-cuda   # NVIDIA GPUs
pip install mistralrs-metal  # Apple Silicon
pip install mistralrs        # CPU only

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

0.8.23

0.8.22

0.8.21

0.8.20

0.8.18

0.8.17

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

0.8.9

0.8.8

0.8.6

0.8.5

This version

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.4-cp310-abi3-win_amd64.whl (20.1 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.4-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23.2 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.4-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (21.7 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.4-cp310-abi3-macosx_11_0_arm64.whl (36.3 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.4-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.4-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.1 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.4-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 1332149be41a57a3b3dd8cd5a8018841a41371bc6c168759a7ef609108cd0049
MD5 58191dab0847709117acf4cc8f8415ad
BLAKE2b-256 9404c3ef5ac3323f873e57a132b9cdeeb777e7125ae91b07247f1ad70348af89

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.4-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.4-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.4-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 781744d76ecba55b4febfa2eeecca1a50a5eab14ba5b2d45334629ea9ae114a9
MD5 b08b3a7f9b7f7487cbbe0c20c74988c9
BLAKE2b-256 b8668ce4bdf92ff240f419dfadac7c4aa5f9efcc8e4778bfbab48c7eb0b02b57

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.4-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.4-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.4-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ccee9aaab4fe30e076e2593b1a2a222aa0fd02da36a1d56264d5597ed6d01f63
MD5 660dbb9cf2b9bbfee8622fc4920f5b78
BLAKE2b-256 6deec6d4744a052a609e52029e088074a72b3dee050011d74a34312c2e2a402c

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.4-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.4-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.4-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b7938139edb0667ab538b0ac48366b08b0660ae515b396ab952668a42fa1cc7d
MD5 a83ac3e442fad2da397773cbc68e10d6
BLAKE2b-256 f053c8ce11cf3134541aeb004e5a46338b2f007b8b5140575d9084739b859cd0

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.4-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page