This release is a pre-release and may not be stable for production use.
mlx-beam
B.E.A.M. — Batched Engine for Apple Metal. Light and modular inference engine, built on MLX.
Work in progress. See the status table below and the changelog.
Install
uv tool install mlx-beam
beam doctor
Inside a uv project: uv add mlx-beam, then uv run beam doctor.
beam doctor reports the Python, MLX, device and memory it sees (--json for scripts) and exits non-zero when MLX is missing or fails to load. It is the only command so far.
What sets it apart
- Robust prefix cache — RAM and SSD tiers, checkpoints for hybrid models. Survives model swaps and restarts.
- Expert streaming — Mixture-of-experts models larger than memory. Residency configurable, from minimal RAM to fully resident.
- No bloat — The core is the token path. Vision, audio, conversion, structured output and tool-call repair are optional extras.
- Batched MTP — Multi-token prediction stays on with many requests at once.
- Batched vision — Images go through the same scheduler; no request waits behind a picture.
- No stalls — A short request beside a long prefill answers in seconds.
- Mixed-precision KV cache — Bits per layer, set at conversion.
- Thinking budget — A hard cap on the reasoning trace, per request.
- Responses API — Next to chat completions, stateless.
The engine reads standard MLX checkpoints and the B.E.A.M. package layout (extras/ next to the shards; see the model cards under huggingface.co/p4ik).
Why it exists
Existing MLX servers either stop at the basics or grow things that have no place in an inference engine: a built-in game, a cloud path that arrives with an update. The ones we ran daily also had bugs where it matters most: prefix cache, batching under load, vision. B.E.A.M. keeps the core to the token path and fixes those paths at the source. Everything else is an extra you choose to install; nothing ever ships in the core that you did not ask for.
Status
| Piece | State |
|---|---|
| CLI, packaging, CI | skeleton |
| Vendored mlx-lm base | planned |
| Prefix cache with recurrent-state checkpoints | planned |
| Multi-token prediction in the batch | planned |
| Expert streaming from SSD | planned |
Measured numbers are published as they are measured, with machine, model and date.
Contributing
See CONTRIBUTING.md. Rules for coding agents are in AGENTS.md.
License
Apache-2.0. Vendored components keep their own licenses; see NOTICE.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file mlx_beam-0.1.0.dev6.tar.gz.
File metadata
- Download URL: mlx_beam-0.1.0.dev6.tar.gz
- Upload date:
- Size: 9.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6f8a7c8b6668ccd37fb276a7e2a952721a0dcdee893c17ca9660a09e987b4951
|
|
| MD5 |
1355701c37f5d6c0292b9b9b6b7dd1d7
|
|
| BLAKE2b-256 |
4816289928402b21e1b2f11ac1eb7663b6f8cdf6eaa10bbb0fb93cb8958470e6
|
Provenance
The following attestation bundles were made for mlx_beam-0.1.0.dev6.tar.gz:
Publisher:
release.yml on p4ik/mlx-beam
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
mlx_beam-0.1.0.dev6.tar.gz -
Subject digest:
6f8a7c8b6668ccd37fb276a7e2a952721a0dcdee893c17ca9660a09e987b4951 - Sigstore transparency entry: 2835248346
- Sigstore integration time:
-
Permalink:
p4ik/mlx-beam@46cc28032d90234d2bdf8efbdaff96c5ab3ad17d -
Branch / Tag:
refs/heads/main - Owner: https://github.com/p4ik
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@46cc28032d90234d2bdf8efbdaff96c5ab3ad17d -
Trigger Event:
workflow_dispatch
-
Statement type:
File details
Details for the file mlx_beam-0.1.0.dev6-py3-none-any.whl.
File metadata
- Download URL: mlx_beam-0.1.0.dev6-py3-none-any.whl
- Upload date:
- Size: 9.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
7c257931b8210d371d1c788e2d232ce5741141b866a70c7dd7fe01cc57a95f5a
|
|
| MD5 |
3767692aa708da5bbe3b6e6c936f589d
|
|
| BLAKE2b-256 |
0e5153701de04b8505462eed576a2a8c39754d8815c09e453601446ce127cc7c
|
Provenance
The following attestation bundles were made for mlx_beam-0.1.0.dev6-py3-none-any.whl:
Publisher:
release.yml on p4ik/mlx-beam
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
mlx_beam-0.1.0.dev6-py3-none-any.whl -
Subject digest:
7c257931b8210d371d1c788e2d232ce5741141b866a70c7dd7fe01cc57a95f5a - Sigstore transparency entry: 2835248425
- Sigstore integration time:
-
Permalink:
p4ik/mlx-beam@46cc28032d90234d2bdf8efbdaff96c5ab3ad17d -
Branch / Tag:
refs/heads/main - Owner: https://github.com/p4ik
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@46cc28032d90234d2bdf8efbdaff96c5ab3ad17d -
Trigger Event:
workflow_dispatch
-
Statement type:
