Fast, flexible LLM inference.

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers
Report project as malware

Project description

mistralrs Python SDK

mistralrs is the Python SDK for mistral.rs, a blazing-fast LLM inference engine.

Documentation

For full documentation, see:

Quick Install

pip install mistralrs-cuda   # NVIDIA GPUs
pip install mistralrs-metal  # Apple Silicon
pip install mistralrs        # CPU only

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for EricLBuehler from gravatar.com EricLBuehler

Unverified details

These details have not been verified by PyPI
Meta
  • License: MIT License (MIT)
  • Author: Eric Buehler
  • Tags llm , inference , machine-learning , ai , transformers
  • Requires: Python >=3.10
Classifiers

Release history Release notifications | RSS feed

0.8.23

0.8.22

0.8.21

0.8.20

0.8.18

0.8.17

0.8.16

0.8.15

0.8.14

0.8.13

0.8.12

This version

0.8.9

0.8.8

0.8.6

0.8.5

0.8.4

0.8.0

0.7.0

0.6.0

0.5.0

0.4.0

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.5

0.2.4

0.2.3

0.2.2

0.2.1

0.2.0

0.1.24

0.1.23

0.1.22

0.1.21

0.1.20

0.1.19

0.1.18

0.1.17

0.1.16

0.1.15

0.1.14

0.1.13

0.1.12

0.1.11

0.1.10

0.1.8

0.1.7

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

mistralrs-0.8.9-cp310-abi3-win_amd64.whl (19.9 MB view details)

Uploaded CPython 3.10+Windows x86-64

mistralrs-0.8.9-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

mistralrs-0.8.9-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (22.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

mistralrs-0.8.9-cp310-abi3-macosx_11_0_arm64.whl (36.2 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

File details

Details for the file mistralrs-0.8.9-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: mistralrs-0.8.9-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 19.9 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for mistralrs-0.8.9-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 903442afa0e97be8b1e8b7f6443f53b1b8c15a6c91be49429d33d504e66631f4
MD5 43eb6989413545cc2cedffc337e10f2a
BLAKE2b-256 5c4ff2811f94c1f99a36eefc22cb0b416d74e24bb6c9b2807a03ffa7ba409880

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.9-cp310-abi3-win_amd64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.9-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.9-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bb3d8786f25d86858f9cda8c1b4a2bff64e2e22616cd3439c2c4f2c5b1f8eb6b
MD5 67f24ba6a9f0e5c6082a263e97c8bd9c
BLAKE2b-256 4893dbce09a6d28e95797270a796b4c8013b07caee1a6743dd39e0d126ce5c25

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.9-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.9-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.9-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 e9e7799d64225d4a36e64ce5089ef5b38dea84c4ef6936f824e6a122409507be
MD5 bb6d636b9c02a0d2ed58a2b874c4e9f6
BLAKE2b-256 f2f200842be60a105dc65fe7f820f96ca9d84d0953f83ddd2bbcda8eacbdcd97

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.9-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file mistralrs-0.8.9-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for mistralrs-0.8.9-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a7e2f0df28004534ea86577b3f4837f8f027339663326d3e7ea98e2567faa8fe
MD5 7750f7e01bf9247d0967a82d197c5acc
BLAKE2b-256 b7467260f21131f25ec15e0360c54af0a64f6590f747b4d00a88789f7cb48090

See more details on using hashes here.

Provenance

The following attestation bundles were made for mistralrs-0.8.9-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on EricLBuehler/mistral.rs

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page