This release is a pre-release and may not be stable for production use.
mlx-beam
B.E.A.M. — Batched Engine for Apple Metal. An MLX inference engine for hybrid-attention and mixture-of-experts language models on Apple silicon.
Work in progress. See the status table below and the changelog.
What it is for
Models like Qwen3.8 (Gated DeltaNet + attention) and Qwen3.8-Flash-Next (125B-A6B MoE with an on-disk n-gram table) do not fit the assumptions of a plain transformer server: their recurrent state cannot be trimmed like a KV cache, their experts do not fit in memory, and speculative decoding under batching needs a rollback that most engines lack. B.E.A.M. is built around those cases:
- a prefix cache with recurrent-state checkpoints, so a long conversation resumes from disk instead of re-prefilling;
- multi-token prediction inside continuous batching;
- expert streaming from SSD with a resident pool, for models larger than RAM;
- a small, explicit OpenAI-compatible API, with everything else as optional extras.
The engine reads standard MLX checkpoints and the B.E.A.M. package layout (extras/ next to the shards; see the model cards under huggingface.co/p4ik).
Install
uv tool install "mlx-beam @ git+https://github.com/p4ik/mlx-beam"
beam doctor
Inside an existing uv project: uv add "mlx-beam @ git+https://github.com/p4ik/mlx-beam", then uv run beam doctor.
beam doctor prints the Python, MLX, device and memory it sees (--json for scripts) and exits non-zero when MLX is missing or fails to load. It is the only command so far.
Status
| Piece | State |
|---|---|
| CLI, packaging, CI | skeleton |
| vendored mlx-lm base | planned |
| prefix cache with recurrent-state checkpoints | planned |
| MTP in the batch | planned |
| expert streaming | planned |
Contributing
See CONTRIBUTING.md. Working rules for people and agents are in AGENTS.md.
License
Apache-2.0. Vendored components keep their own licenses; see NOTICE and VENDORED.md.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file mlx_beam-0.1.0.dev5.tar.gz.
File metadata
- Download URL: mlx_beam-0.1.0.dev5.tar.gz
- Upload date:
- Size: 8.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8038bf7c4c3b4494d7ab26df8342eaa9e3c183f2564766e42d7b0f802d51563b
|
|
| MD5 |
bf01d0b9480c428fdda3f71d7d8741e8
|
|
| BLAKE2b-256 |
dc2042ca91e031de8ac019d837d20c3262bd552a7f62a294f760168062f734c5
|
Provenance
The following attestation bundles were made for mlx_beam-0.1.0.dev5.tar.gz:
Publisher:
release.yml on p4ik/mlx-beam
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
mlx_beam-0.1.0.dev5.tar.gz -
Subject digest:
8038bf7c4c3b4494d7ab26df8342eaa9e3c183f2564766e42d7b0f802d51563b - Sigstore transparency entry: 2834959174
- Sigstore integration time:
-
Permalink:
p4ik/mlx-beam@caff7bc8b806e0c058b0cf6913b8ebcbdbc01ad8 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/p4ik
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@caff7bc8b806e0c058b0cf6913b8ebcbdbc01ad8 -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file mlx_beam-0.1.0.dev5-py3-none-any.whl.
File metadata
- Download URL: mlx_beam-0.1.0.dev5-py3-none-any.whl
- Upload date:
- Size: 9.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
be8d6d708c44a1a8b2330a893e0303899f8a25b971b6b4cdc9851326b591c502
|
|
| MD5 |
83d10d1bdf03eb32cb76c128c00f2d39
|
|
| BLAKE2b-256 |
3408d7abe0159164dfcb3ea4b9fd3e3f8d9a11ea047419f2f632991ecef712b3
|
Provenance
The following attestation bundles were made for mlx_beam-0.1.0.dev5-py3-none-any.whl:
Publisher:
release.yml on p4ik/mlx-beam
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
mlx_beam-0.1.0.dev5-py3-none-any.whl -
Subject digest:
be8d6d708c44a1a8b2330a893e0303899f8a25b971b6b4cdc9851326b591c502 - Sigstore transparency entry: 2834959239
- Sigstore integration time:
-
Permalink:
p4ik/mlx-beam@caff7bc8b806e0c058b0cf6913b8ebcbdbc01ad8 -
Branch / Tag:
refs/heads/main - Owner: https://github.com/p4ik
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@caff7bc8b806e0c058b0cf6913b8ebcbdbc01ad8 -
Trigger Event:
workflow_dispatch
-
Statement type:
