mashbox 0.1.8

MASHBOX Labs makes open models run faster and cheaper on TPUs and accelerated hardware, as well as exploring the future of the intersection of AI and gaming.

Quickstart • Models • Docs • Hugging Face • Coding agents

Qwen3.8-27B on one TPU v6e chip: first-time install to first answer (sped up 10x).

Quickstart

From your own machine, with the Google Cloud CLI set up (docs/google-cloud.md does that from a new Google account). This runs Qwen3.8-27B on one TPU v6e chip:

gcloud compute tpus tpu-vm create mashbox-1 --zone=us-east5-b --accelerator-type=v6e-1 --version=v2-alpha-tpuv6e
gcloud compute tpus tpu-vm ssh mashbox-1 --zone=us-east5-b --command='python3 -m pip install uv && python3 -m uv tool install --python 3.12 "mashbox[tpu]" && ~/.local/bin/mashbox run qwen3-8-27b --prompt "Write a haiku about the sea."'
gcloud compute tpus tpu-vm delete mashbox-1 --zone=us-east5-b --quiet

The chip's system Python is 3.10 and JAX for TPU needs 3.12, so the second line installs uv, which brings its own Python 3.12 and puts the mashbox command in ~/.local/bin. The first run downloads the 26.9 GB weights file and the compile cache; with them on disk, the server on a TPU v6e-1 is ready in 69.6 s. If the create command says the zone has no capacity, use another zone from step 6 of the setup guide in all three lines.

The chip bills from create to delete: one TPU v6e chip is $2.70 an hour at Google's on-demand list price. To serve Qwen3.8-27B with an OpenAI-compatible API instead, see docs/models/qwen3-8-27b.md.

Models

26 models and files run in mashbox: 3 MASHBOX builds and 23 GGUF files of 18 models.

MASHBOX builds

Models we port or make ourselves. A BSHJ build is our weights file for the model, run on our TPU kernels. Measured is each one's number against its baseline, on the chip named.

GGUF files that run in our engine

GGUF files from other publishers, run as they are by mashbox. Level and Runs on say how far each file has been checked; a speed shows only where a chip run measured it on our kernels.

Full catalog, with the run command for each file.

Every number names the chip it was measured on; each model's guide has all its results. Options for any model: ~/.local/bin/mashbox run <model> --help.

Results

Each chart names the hardware on both sides.

_{Qwen3-30B-A3B is not in this package: this chart compares quantized files of it, each scored the same way.}

What you get

• One command per model. mashbox run <model> downloads the weights, checks every file's sha256, loads the model and serves it.
• Fits on fewer chips. Qwen3.8-27B runs on a single TPU v6e chip (32 GB) from a 26.94 GB weights file at 8.01 bits per weight, with 99.72% top-1 agreement against Qwen's own bf16 weights.
• Fast starts. Models ship their compiled TPU kernels, and Qwen3.8-27B also ships a precompiled compile cache for TPU v6e, so a start skips compiling: Qwen3.8-27B on TPU v6e-1 is ready in 69.6 s with it, 326.6 s without it.
• An OpenAI-compatible server for the language models, with batching: 8 requests at once share the chip.
• Works with Claude Code, Codex and Pi. The server also speaks the Anthropic Messages and OpenAI Responses APIs, so coding agents can use a model you serve as their LLM: Use with coding agents.
• Docs for coding agents. AGENTS.md gives an agent the rules and the steps to set up a chip, run a model and delete the chip; llms.txt lists the docs.
• Named files. Every weights file on Hugging Face is named by its type (for example Qwen3.8-27B-BQ8_R.bshj), with size, bits per weight and measured quality next to it.

TPU v6e and v5e

v6e is the default. A model too big for one v5e chip (16 GB) splits across the 4 chips of a v5e-4 when its row says so; the models table says which chips each model runs on. Specs from Google's v6e and v5e pages. More in docs/chips.md.

TPU v6e (one chip and a v6e-4) against one NVIDIA H100 at Google Cloud list prices, with the memory bandwidth of each and the cost per 1M output tokens of Qwen3.8-27B (27B parameters, dense): TPU or GPU: price and bandwidth.

Open code, compiled kernels

The source of the model ports, CLI, runtime, demo apps and docs is in this repo, under Apache-2.0. Each model ships with its own compiled TPU kernels (our Culet kernels) and, where we built one, a precompiled compile cache, only what that model needs: mashbox run downloads them from the model's Hugging Face repo and checks them against the sha256 in models.json. Models that list plain JAX as tested also run without them, slower. The compiled files are under the MASHBOX Kernel Licence: free for personal use, research and evaluation, and for any use by companies under US$1M annual revenue. Larger companies in production: hello@mashbox.gg.

Coding agents

Use a model as your agent's LLM. Claude Code, Codex and Pi work with every LLM that mashbox run serves: start the server, open the tunnel, set the agent's base URL. Settings for each agent and measured results: Use with coding agents.

Let an agent set it up. AGENTS.md gives an agent the rules and the steps, from a Google account to a run and a deleted chip, and tells it to ask you before anything that costs money. llms.txt lists the docs.

Community and links

Licence

Our code is under the Apache License 2.0. The compiled kernels and compile cache are under the MASHBOX Kernel Licence. Model weights keep each model's own licence, shown on its Hugging Face card and kept unchanged in licenses/. Details: docs/licence.md.

Citation

@software{mashbox,
  author = {{MASHBOX Labs}},
  title = {mashbox: open models that run faster and cheaper},
  url = {https://github.com/mashbox-labs/mashbox},
  year = {2026}
}

Thank you

• Tencent Hunyuan for Hy-Nomas and Qwen for Qwen3.8-27B, and for releasing their weights.
• The JAX and Pallas teams, and Google Cloud TPU.
• llama.cpp and the ggml authors for GGUF; mashbox reads GGUF files with their gguf package (GGUF files).
• Hugging Face for hosting the weights and uv for a one-line Python 3.12.