Skip to main content

mistral.rs

mistralrs is a Python package which provides an easy to use API for mistral.rs.

Example

More examples can be found here!

from mistralrs import Runner, Which, ChatCompletionRequest

runner = Runner(
    which=Which.Plain(
        model_id="microsoft/Phi-3.5-mini-instruct",
    ),
    in_situ_quant="Q4K",
)

res = runner.send_chat_completion_request(
    ChatCompletionRequest(
        model="mistral",
        messages=[
            {"role": "user", "content": "Tell me a story about the Rust type system."}
        ],
        max_tokens=256,
        presence_penalty=1.0,
        top_p=0.1,
        temperature=0.1,
    )
)
print(res.choices[0].message.content)
print(res.usage)

Multimodal (audio + image) example

mistralrs also supports multimodal vision models that can reason over both images and audio clips via the same OpenAI-style audio_url / image_url format. The example below queries the Phi-4-Multimodal model with a single image and an audio recording – notice how the text prompt references them via <|audio_1|> and <|image_1|> tokens (indexing starts at 1):

from mistralrs import Runner, Which, ChatCompletionRequest, VisionArchitecture

runner = Runner(
    which=Which.VisionPlain(
        model_id="microsoft/Phi-4-multimodal-instruct",
        arch=VisionArchitecture.Phi4MM,
    ),
)

IMAGE_URL = "https://www.allaboutbirds.org/guide/assets/og/528129121-1200px.jpg"
AUDIO_URL = "https://upload.wikimedia.org/wikipedia/commons/4/42/Bird_singing.ogg"

response = runner.send_chat_completion_request(
    ChatCompletionRequest(
        model="phi4mm",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "audio_url", "audio_url": {"url": AUDIO_URL}},
                    {"type": "image_url", "image_url": {"url": IMAGE_URL}},
                    {
                        "type": "text",
                        "text": "<|audio_1|><|image_1|> Describe in detail what is happening, referencing both what you hear and what you see.",
                    },
                ],
            }
        ],
        max_tokens=256,
        temperature=0.2,
        top_p=0.9,
    )
)

print(response.choices[0].message.content)

See examples/python/phi4mm_audio.py for a ready-to-run version.

Please find API docs here and the type stubs here, which are another great form of documentation.

We also provide a cookbook here!

Metadata

Release files for mistralrs 0.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for mistralrs 0.6.0
File Actions
mistralrs-0.6.0-cp312-none-win_amd64.whl CPython 3.12 none Windows x86-64 Details for mistralrs-0.6.0-cp312-none-win_amd64.whl
mistralrs-0.6.0-cp312-cp312-manylinux_2_39_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.39+ x86-64 Details for mistralrs-0.6.0-cp312-cp312-manylinux_2_39_x86_64.whl
mistralrs-0.6.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.17+ x86-64 Details for mistralrs-0.6.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
mistralrs-0.6.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.12 CPython 3.12 Linux glibc 2.17+ ARM64 Details for mistralrs-0.6.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
mistralrs-0.6.0-cp312-cp312-macosx_11_0_arm64.whl CPython 3.12 CPython 3.12 macOS 11.0+ ARM64 Details for mistralrs-0.6.0-cp312-cp312-macosx_11_0_arm64.whl
mistralrs-0.6.0-cp311-none-win_amd64.whl CPython 3.11 none Windows x86-64 Details for mistralrs-0.6.0-cp311-none-win_amd64.whl
mistralrs-0.6.0-cp311-cp311-manylinux_2_39_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.39+ x86-64 Details for mistralrs-0.6.0-cp311-cp311-manylinux_2_39_x86_64.whl
mistralrs-0.6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.17+ x86-64 Details for mistralrs-0.6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
mistralrs-0.6.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.11 CPython 3.11 Linux glibc 2.17+ ARM64 Details for mistralrs-0.6.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
mistralrs-0.6.0-cp311-cp311-macosx_11_0_arm64.whl CPython 3.11 CPython 3.11 macOS 11.0+ ARM64 Details for mistralrs-0.6.0-cp311-cp311-macosx_11_0_arm64.whl
mistralrs-0.6.0-cp310-none-win_amd64.whl CPython 3.10 none Windows x86-64 Details for mistralrs-0.6.0-cp310-none-win_amd64.whl
mistralrs-0.6.0-cp310-cp310-manylinux_2_39_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.39+ x86-64 Details for mistralrs-0.6.0-cp310-cp310-manylinux_2_39_x86_64.whl
mistralrs-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.17+ x86-64 Details for mistralrs-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
mistralrs-0.6.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.10 CPython 3.10 Linux glibc 2.17+ ARM64 Details for mistralrs-0.6.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
mistralrs-0.6.0-cp310-cp310-macosx_11_0_arm64.whl CPython 3.10 CPython 3.10 macOS 11.0+ ARM64 Details for mistralrs-0.6.0-cp310-cp310-macosx_11_0_arm64.whl

Total release size: 257.3 MB

Release files / mistralrs-0.6.0-cp312-none-win_amd64.whl

Download URL mistralrs-0.6.0-cp312-none-win_amd64.whl
Size 16.0 MB
Tags CPython 3.12 Windows x86-64
SHA-256 checksum
How to use checksums
34a7490062470f54d69e616ac4f816784fc3009f964e0b63c23bb03fe47ef096
BLAKE2b-256 checksum
How to use checksums
b38e9093523e6f9df6200e58c6b2a48b9ad89c7468879e42692c7db71b7cd649
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp312-cp312-manylinux_2_39_x86_64.whl

Download URL mistralrs-0.6.0-cp312-cp312-manylinux_2_39_x86_64.whl
Size 16.2 MB
Tags CPython 3.12 Linux glibc 2.39+ x86-64
SHA-256 checksum
How to use checksums
f5fc680b1495837bf5224c39856abb60eb9040b85c454b081aba84101869e41c
BLAKE2b-256 checksum
How to use checksums
15164db6e232d70306973666242a386520aa79e85beb47a79a1de6eea5599c19
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Download URL mistralrs-0.6.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size 19.1 MB
Tags CPython 3.12 Linux glibc 2.17+ x86-64
SHA-256 checksum
How to use checksums
617220d1397f76dc6d05d2d209a129c8b1a7cf6ce97085613349533339ef69d5
BLAKE2b-256 checksum
How to use checksums
62e0a581e387cd2be411bf31fc31943f5c48e01d657ee2041254099af054a350
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL mistralrs-0.6.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 18.0 MB
Tags CPython 3.12 Linux glibc 2.17+ ARM64
SHA-256 checksum
How to use checksums
f7ccfccf9f83f0c60dc0ab5721159a7d07f133c98e74901d069fa84e034e1e24
BLAKE2b-256 checksum
How to use checksums
086b93ece004a25cc51d967fa7e8e5fe9223fcacb5daf2a386f631b60386c88f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp312-cp312-macosx_11_0_arm64.whl

Download URL mistralrs-0.6.0-cp312-cp312-macosx_11_0_arm64.whl
Size 16.5 MB
Tags CPython 3.12 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
3d48fe4c29d352341d7a48661105c3d2fedddf07a1d4727b04dee93da32d3fcc
BLAKE2b-256 checksum
How to use checksums
2a4e57f534dd083a7cca5c2ddd6c697c43448bd36d75745291da067556ecccc3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp311-none-win_amd64.whl

Download URL mistralrs-0.6.0-cp311-none-win_amd64.whl
Size 16.0 MB
Tags CPython 3.11 Windows x86-64
SHA-256 checksum
How to use checksums
44f1d3ab23cd4204e3012f57d2d1de33185eaffe69aefb4b587bec2b4dea02a8
BLAKE2b-256 checksum
How to use checksums
32e28662160aad243a7f6fa0cf72dafab6fc4243f8f824942c93a6e508718d3f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp311-cp311-manylinux_2_39_x86_64.whl

Download URL mistralrs-0.6.0-cp311-cp311-manylinux_2_39_x86_64.whl
Size 16.1 MB
Tags CPython 3.11 Linux glibc 2.39+ x86-64
SHA-256 checksum
How to use checksums
62067149d15fd7f446e2a98e45a26ab83c19f854de9373c4897a2b8c9c8be6b5
BLAKE2b-256 checksum
How to use checksums
0935808a00e49b70cb34a08f04b7fe174afbd5c4be6876540cba6daf6dd73d36
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Download URL mistralrs-0.6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size 19.1 MB
Tags CPython 3.11 Linux glibc 2.17+ x86-64
SHA-256 checksum
How to use checksums
a008753239c700136258274a7016f30811103027286f89e8a7cacfbff7b6a0f9
BLAKE2b-256 checksum
How to use checksums
c3b40da6e7c2efded8193e43f9b46f1dfcdffa5bdfab25890cd54320f074456a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL mistralrs-0.6.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 18.0 MB
Tags CPython 3.11 Linux glibc 2.17+ ARM64
SHA-256 checksum
How to use checksums
e2b92e4e78ae95c2306e9a2981bd91219ee1e362a629988822de2e8978239829
BLAKE2b-256 checksum
How to use checksums
a017c80651ca2097287bf144ef256d6e8568cd5337a4d47b7020e1203d09770e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp311-cp311-macosx_11_0_arm64.whl

Download URL mistralrs-0.6.0-cp311-cp311-macosx_11_0_arm64.whl
Size 16.6 MB
Tags CPython 3.11 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
a53636c6a627e07c7b43184fa69a46dbb5e25c76e1c3fc3c2f4186f2b792a175
BLAKE2b-256 checksum
How to use checksums
1c433309937fee55260a74f10fc8c135ef1e3f29733ba81bbd65c94672befb70
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp310-none-win_amd64.whl

Download URL mistralrs-0.6.0-cp310-none-win_amd64.whl
Size 16.0 MB
Tags CPython 3.10 Windows x86-64
SHA-256 checksum
How to use checksums
a81030e3a74f35eff1a3491bd40f2f5d524ab659c6345d8a94101f424e770ede
BLAKE2b-256 checksum
How to use checksums
f5db12f8b4f7e91d94004e1242d65123760b0e4b3e45596880b61cd0bc903502
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp310-cp310-manylinux_2_39_x86_64.whl

Download URL mistralrs-0.6.0-cp310-cp310-manylinux_2_39_x86_64.whl
Size 16.1 MB
Tags CPython 3.10 Linux glibc 2.39+ x86-64
SHA-256 checksum
How to use checksums
069eb40faa19bbddd7373d4b06013aed492fa829d70d876c12dad85202b37121
BLAKE2b-256 checksum
How to use checksums
2f67c09f13bc2264384373be6fb8995e86b41058c4cd913b023aaa4d336b4f54
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Download URL mistralrs-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size 19.1 MB
Tags CPython 3.10 Linux glibc 2.17+ x86-64
SHA-256 checksum
How to use checksums
c75a658f666197ed65cf441c0d23d33063a7037a99d507ebfbf31ad772c15a6a
BLAKE2b-256 checksum
How to use checksums
8b6babba5586ca664174d148d747c0c13a746ae40be6600de65aeec94028c512
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL mistralrs-0.6.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 18.0 MB
Tags CPython 3.10 Linux glibc 2.17+ ARM64
SHA-256 checksum
How to use checksums
8b210b2c925acc191642cbef80e93a3b1b36b5d883f92e2f4dc6cfb322ac31d0
BLAKE2b-256 checksum
How to use checksums
471f0ed71530e39f7dfd005ea5accb4a516dd06de79382a800e6d8c89b5d7d61
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Release files / mistralrs-0.6.0-cp310-cp310-macosx_11_0_arm64.whl

Download URL mistralrs-0.6.0-cp310-cp310-macosx_11_0_arm64.whl
Size 16.6 MB
Tags CPython 3.10 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
6fbbb16dab7c9d21b41d849e40022dff2459d88ce4a3312b05d93730b455662d
BLAKE2b-256 checksum
How to use checksums
f34771e2d421597570fc93edf98398f2d90e40f98fef36c7e7bc87b4208e8624
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.12.7

Security for mistralrs 0.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.6.0 This release

15 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page