Threadstone

Threadstone is offline terminal chat for local MLX language models on Apple Silicon. It starts a local mlx-vlm server for the model you choose, streams the answer into your terminal, keeps the conversation usable across crashes, and shuts the server down when you leave.

The useful part is ownership. The network is only for setup and model downloads. After that, chat runs against local snapshots with Hugging Face offline flags, no telemetry, no hosted API, and no account.

Quick Start

Requirements: macOS, Apple Silicon, Python 3.13+, Homebrew, and about 12 GB for the default model set.

git clone https://github.com/gabrimatic/threadstone.git
cd threadstone
./setup.sh

setup.sh creates ~/mlx-env, installs the MLX runtime, downloads the configured Qwen model snapshots, installs the threadstone CLI, and writes shell helpers for oracle, forge, and quench.

Start a chat:

oracle
oracle 4B
oracle "be terse" 2B

Use the installed CLI directly:

threadstone --list-models
threadstone --doctor
threadstone "answer like a systems engineer" 9B

Local Runtime

Threadstone uses the network during setup. Runtime chat stays on localhost and local model files.

Offline guard:

HF_HUB_OFFLINE=1
TRANSFORMERS_OFFLINE=1
HF_DATASETS_OFFLINE=1
HF_HUB_DISABLE_TELEMETRY=1
HF_HUB_DISABLE_IMPLICIT_TOKEN=1
HF_TOKEN=
DO_NOT_TRACK=1
DISABLE_TELEMETRY=1
ANONYMIZED_TELEMETRY=0

Commands

Inside chat:

Shell helpers from setup.sh:

Models

Default models are 4-bit MLX snapshots from mlx-community.

Each model has its own default port, memory estimate, context threshold, and response budget in config.py.

Behavior

Threadstone owns the server lifecycle for normal chat.

• Port recovery: if the default port is busy, Threadstone scans forward and starts the model on the next available localhost port.
• RAM guard: startup checks free and reclaimable memory before launching a model, including other reachable model servers.
• Crash recovery: if the server disappears during a turn, Threadstone restarts it and resends the pending message.
• Interrupt safety: Ctrl-C mid-answer keeps the partial output; Ctrl-C while a request is in flight cancels the turn. The chat survives both.
• Thinking models: reasoning streams dimmed until </think>, then the final answer prints normally and only the final answer is sent back in later history.
• Attachments: /read accepts text files and directory listings, rejects non-regular files, rejects binary-looking content, and caps file payloads at 50 KB.
• Context trimming: old turns are trimmed when the approximate context crosses the configured threshold while preserving valid role alternation.

Architecture

oracle / threadstone
    |
    v
threadstone.py
    |-- argument parsing, doctor, REPL, streaming, history
    |-- ServerManager starts and monitors mlx_vlm.server
    |-- /read attaches bounded local text context
    |
    v
config.py
    |-- model registry, ports, limits, memory estimates
    |-- snapshot resolution from the Hugging Face cache
    |-- offline environment guard

Files:

Development

python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
ruff check .
mypy
python3 -m unittest discover -s tests -t . -v
python3 -m build --sdist --wheel
python3 -m twine check --strict dist/*

Run the CLI without starting a model:

threadstone --version
threadstone --list-models

Run the local setup check after ./setup.sh:

threadstone --doctor --all-models

Package Release

Threadstone is a Python package, so the package registry is PyPI, not pub.dev. pub.dev is for Dart and Flutter packages.

Release path:

1. Bump the version in pyproject.toml and VERSION in threadstone.py, and add a CHANGELOG.md entry.
2. Push to main and wait for CI to pass.
3. Publish a GitHub release with a vX.Y.Z tag that matches the package version.
4. The release workflow checks the tag against the package version, builds the sdist and wheel, and publishes to PyPI through Trusted Publishing.

PyPI Trusted Publishing must be configured for:

Security

Runtime chat is local by design, but model files and dependencies still come from external package and model hosts during setup.

Report vulnerabilities through GitHub private vulnerability reporting. Do not open a public issue for security reports.