Skip to main content

mistral.rs

mistralrs is a Python package which provides an easy to use API for mistral.rs.

Example

More examples can be found here!

from mistralrs import Runner, Which, ChatCompletionRequest

runner = Runner(
    which=Which.Plain(
        model_id="microsoft/Phi-3.5-mini-instruct",
    ),
    in_situ_quant="Q4K",
)

res = runner.send_chat_completion_request(
    ChatCompletionRequest(
        model="mistral",
        messages=[
            {"role": "user", "content": "Tell me a story about the Rust type system."}
        ],
        max_tokens=256,
        presence_penalty=1.0,
        top_p=0.1,
        temperature=0.1,
    )
)
print(res.choices[0].message.content)
print(res.usage)

Please find API docs here and the type stubs here, which are another great form of documentation.

We also provide a cookbook here!

Metadata

Release files for mistralrs 0.3.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for mistralrs 0.3.4
File Actions
mistralrs-0.3.4-cp312-none-win_amd64.whl CPython 3.12 none Windows x86-64 Details for mistralrs-0.3.4-cp312-none-win_amd64.whl
mistralrs-0.3.4-cp312-cp312-manylinux_2_39_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.39+ x86-64 Details for mistralrs-0.3.4-cp312-cp312-manylinux_2_39_x86_64.whl
mistralrs-0.3.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.12 CPython 3.12 Linux glibc 2.17+ ARM64 Details for mistralrs-0.3.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
mistralrs-0.3.4-cp312-cp312-macosx_11_0_arm64.whl CPython 3.12 CPython 3.12 macOS 11.0+ ARM64 Details for mistralrs-0.3.4-cp312-cp312-macosx_11_0_arm64.whl
mistralrs-0.3.4-cp311-none-win_amd64.whl CPython 3.11 none Windows x86-64 Details for mistralrs-0.3.4-cp311-none-win_amd64.whl
mistralrs-0.3.4-cp311-cp311-manylinux_2_39_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.39+ x86-64 Details for mistralrs-0.3.4-cp311-cp311-manylinux_2_39_x86_64.whl
mistralrs-0.3.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.11 CPython 3.11 Linux glibc 2.17+ ARM64 Details for mistralrs-0.3.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
mistralrs-0.3.4-cp311-cp311-macosx_11_0_arm64.whl CPython 3.11 CPython 3.11 macOS 11.0+ ARM64 Details for mistralrs-0.3.4-cp311-cp311-macosx_11_0_arm64.whl
mistralrs-0.3.4-cp310-none-win_amd64.whl CPython 3.10 none Windows x86-64 Details for mistralrs-0.3.4-cp310-none-win_amd64.whl
mistralrs-0.3.4-cp310-cp310-manylinux_2_39_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.39+ x86-64 Details for mistralrs-0.3.4-cp310-cp310-manylinux_2_39_x86_64.whl
mistralrs-0.3.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.10 CPython 3.10 Linux glibc 2.17+ ARM64 Details for mistralrs-0.3.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
mistralrs-0.3.4-cp310-cp310-macosx_11_0_arm64.whl CPython 3.10 CPython 3.10 macOS 11.0+ ARM64 Details for mistralrs-0.3.4-cp310-cp310-macosx_11_0_arm64.whl

Total release size: 152.6 MB

Release files / mistralrs-0.3.4-cp312-none-win_amd64.whl

Download URL mistralrs-0.3.4-cp312-none-win_amd64.whl
Size 10.6 MB
Tags CPython 3.12 Windows x86-64
SHA-256 checksum
How to use checksums
1e2e2029695b0f89900307bda41daa5c40e3f476efb94f81a760d189d232fcd7
BLAKE2b-256 checksum
How to use checksums
fddc743a1f379d9711c09e42c5bcd882c0400ef705aae9e510f6b6e8265b6b2a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp312-cp312-manylinux_2_39_x86_64.whl

Download URL mistralrs-0.3.4-cp312-cp312-manylinux_2_39_x86_64.whl
Size 15.2 MB
Tags CPython 3.12 Linux glibc 2.39+ x86-64
SHA-256 checksum
How to use checksums
8e848e8160fd04b29aec4eea65857bf148e44ad95927a9264afbc0bfb5bcbf8d
BLAKE2b-256 checksum
How to use checksums
6cbde5045f13eb65431e157df285012da2c6b4b96db099fa61cd900e737118d4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL mistralrs-0.3.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 14.2 MB
Tags CPython 3.12 Linux glibc 2.17+ ARM64
SHA-256 checksum
How to use checksums
e6e7adb1488760a4c73a9f9842752ff3c0522f240f00336cfc3a98fa48ef171b
BLAKE2b-256 checksum
How to use checksums
54a7d8b214e7e658a23feb9de9491da8cb69050224da55c238c3a1da1b205441
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp312-cp312-macosx_11_0_arm64.whl

Download URL mistralrs-0.3.4-cp312-cp312-macosx_11_0_arm64.whl
Size 10.9 MB
Tags CPython 3.12 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
52051d8cdd5061193de3ea270790ea41202c99aa6abe898f27efa2ac5a0d1c2d
BLAKE2b-256 checksum
How to use checksums
6e96c642ddfca1315126fb9c505f017f3d20ebd091a2c4fde402c6275560fa03
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp311-none-win_amd64.whl

Download URL mistralrs-0.3.4-cp311-none-win_amd64.whl
Size 10.6 MB
Tags CPython 3.11 Windows x86-64
SHA-256 checksum
How to use checksums
1fd9789a7432962e7675837fd31e0429977dfd802ff7ea491235bcd3f6c4435c
BLAKE2b-256 checksum
How to use checksums
cabde322565da6331d4c9c276454b312699320039a02a7c70110c012e72720ce
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp311-cp311-manylinux_2_39_x86_64.whl

Download URL mistralrs-0.3.4-cp311-cp311-manylinux_2_39_x86_64.whl
Size 15.2 MB
Tags CPython 3.11 Linux glibc 2.39+ x86-64
SHA-256 checksum
How to use checksums
4b1204c21585dc460f571ad3779d8323120d91cd21b4503ac9c8d3fa53d72eda
BLAKE2b-256 checksum
How to use checksums
b8bd76be83905f224ac48b7309020194d9c65c410faaff1438794fcae5820bd0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL mistralrs-0.3.4-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 14.2 MB
Tags CPython 3.11 Linux glibc 2.17+ ARM64
SHA-256 checksum
How to use checksums
68c9cc3530340954175f1b8a67d22ac9968a37bb91be67c177265a990c842ba3
BLAKE2b-256 checksum
How to use checksums
2ac18fc60bb01354611736d3bc33acb7466af986f156f94126be485485eec0d8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp311-cp311-macosx_11_0_arm64.whl

Download URL mistralrs-0.3.4-cp311-cp311-macosx_11_0_arm64.whl
Size 10.9 MB
Tags CPython 3.11 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
a51f855aafc4d721ef52219644f11c0c2c731e5bfe8c1f3a6eb6b6bb5dd2ec27
BLAKE2b-256 checksum
How to use checksums
ff9ff452cc9e91a471530c6c87ac2e20bd7fa744a830f4851acae58ea6b54459
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp310-none-win_amd64.whl

Download URL mistralrs-0.3.4-cp310-none-win_amd64.whl
Size 10.6 MB
Tags CPython 3.10 Windows x86-64
SHA-256 checksum
How to use checksums
847c8f0c3af2a8a47ab0084bbac2eb91e42f98ea44441444a7caa0315827f83a
BLAKE2b-256 checksum
How to use checksums
6d8478a7b4812b0f6a0c8f8b9215109ade0eb3895eca7a4b6a59e817eb8be30c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp310-cp310-manylinux_2_39_x86_64.whl

Download URL mistralrs-0.3.4-cp310-cp310-manylinux_2_39_x86_64.whl
Size 15.2 MB
Tags CPython 3.10 Linux glibc 2.39+ x86-64
SHA-256 checksum
How to use checksums
488c9d3ab0ea3da4e88c94a81b917a64b60e28c417e595355a4323a5addd2bd8
BLAKE2b-256 checksum
How to use checksums
9573b41e49982730aa6c015daf8100bf42958d46369d739f1bcc978a046be24d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL mistralrs-0.3.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 14.2 MB
Tags CPython 3.10 Linux glibc 2.17+ ARM64
SHA-256 checksum
How to use checksums
6360b20daf20f62c7698d94acef31f037598bdf4566fa0d684bb4a1de62b435d
BLAKE2b-256 checksum
How to use checksums
0922809bcd4e763ac21308776f7832c9f6fc323ed8c11fc6b46df22a032c9c8a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Release files / mistralrs-0.3.4-cp310-cp310-macosx_11_0_arm64.whl

Download URL mistralrs-0.3.4-cp310-cp310-macosx_11_0_arm64.whl
Size 10.9 MB
Tags CPython 3.10 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
9504782ecec12faf75f1ebb097f6e586d22c67cf80c9b3376c89748f925ed8f7
BLAKE2b-256 checksum
How to use checksums
9153f37e5fcc54648cf2bcbbdb6d683e2e595975c841e9fee80b4dd65ac780dc
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.1.1 CPython/3.12.7

Security for mistralrs 0.3.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.3.4 This release

12 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page