yaarbal 0.1.4

BitTorrent-inspired peer-to-peer distributed LLM inference

Yaarbal

Yaarbal — distributed LLM inference over a credit-metered P2P network.

Status: early prototype. The credit economy and trust system are demo-grade; do not run production workloads on the public network.

Every node hosts a complete model. A tracker brokers peer discovery. Inference requests are routed to available seeders, credits are earned by hosting and spent by querying. No central GPU pool — just peers.

Visit yaarbal.app for the demo network.

Quickstart

pip install yaarbal
yaarbal run qwen2.5-0.5b-instruct "hello"

This auto-discovers seeders on the configured tracker (https://tracker.yaarbal.app by default), submits the prompt, and streams tokens to your terminal. If no seeder is reachable, the command exits with a clear error and tells you how to start one.

Installation

From PyPI

pip install yaarbal
yaarbal --version

See docs/distribution/INSTALL.md for Docker and Homebrew paths.

From source (internal)

git clone <yaarbal repo> && cd yaarbal
pip install -e ".[dev]"

Requirements

• Python 3.11 or newer
• macOS or Linux (Windows is untested)
• (Optional) HF_TOKEN for gated models such as Llama 3.1 8B and above

CPU-only works for small models (Qwen 0.5B, Phi-3-mini Q4_K_M, MiniGPT). GPU acceleration (CUDA / Apple MPS) is auto-detected; pass --device cpu to force CPU.

Usage

yaarbal is a single binary with subcommands. Run yaarbal --help for the full list.

One-shot inference

yaarbal run qwen2.5-0.5b-instruct "your prompt here"

Streams tokens to stdout. Discovers peers via the tracker; falls back to pull-mode if the seeder does not support SSE streaming. Configuration precedence: explicit flags > YAARBAL_* env vars > ~/.yaarbal/config.toml > built-in defaults.

Host a seeder

yaarbal seed --model qwen2.5-0.5b-instruct

Loads the named model end-to-end, registers with the tracker, and serves /inference, /inference/stream, /v1/chat/completions, and /v1/models over HTTP. Seeders earn credits on every settled inference and spend them when running yaarbal run against the network.

Single-node mode (local, no tracker)

# Terminal 1
yaarbal seeder --mode single-node --model qwen2.5-0.5b-instruct --port 7001

# Terminal 2
YAARBAL_CHAT_BACKEND=single YAARBAL_SINGLE_NODE_URL=http://127.0.0.1:7001 \
  yaarbal chat --model qwen2.5-0.5b-instruct

Best for local development and offline use. See QUICKSTART.md for the full walkthrough.

Interactive chat

yaarbal chat --model qwen2.5-0.5b-instruct

Multi-turn REPL with rolling conversation history. In-REPL commands: /models, /health, /metrics, /reset, /quit.

Other subcommands

Command	Effect
yaarbal status	Credit balance, hosted models, tracker peer count (--json for scripting)
yaarbal tracker [PORT]	Run a tracker for a self-hosted Yaarbal network (default port 8000)
yaarbal ui	Serve the pre-built browser chat UI from web/dist/ on 127.0.0.1:5173 (auto-opens browser; --no-open to suppress; requires cd web && npm run build first)
yaarbal --version	Print the installed package version
yaarbal --help	Full subcommand list

Browser UI (development)

To run the React frontend in dev mode:

cd web && npm install && npm run dev

Open http://localhost:5173. Configure the backend mode, model, and URLs in the Connection panel.

Optional swarm authentication

Set YAARBAL_AUTH_USER and YAARBAL_AUTH_PASS before starting the tracker, seeders, and leechers to enable HTTP Basic Auth across all HTTP traffic. If unset, authentication is disabled. HTTP Basic is not a substitute for TLS — see KNOWN_LIMITATIONS.md.

Docker

Role-separated services are defined in docker-compose.yml. The fastest path mirrors the single-node quickstart:

docker compose --profile single-node up single-node
# from a second terminal:
YAARBAL_CHAT_BACKEND=single YAARBAL_SINGLE_NODE_URL=http://127.0.0.1:7001 \
  yaarbal chat --model qwen2.5-0.5b-instruct

Other profiles: default (tracker + minigpt seeders), convenience (all-in-one orchestrator), query (one-shot leecher), web (browser chat proxy). See QUICKSTART.md for the full profile and port matrix.

Testing

pip install -e ".[dev]"
make test                # full suite
make test-unit           # unit tier only
make test-integration    # integration tier only
make test-verbose        # full suite with verbose output
python -m pytest tests/unit/test_tracker.py -v   # one file

CI runs on every push and PR: .github/workflows/python-ci.yml (Python 3.10 syntax check on *.py) and .github/workflows/pr-regression.yml (unittest discover on PRs into main).

Note: pyproject.toml requires Python >=3.11 for installation; the CI syntax-check workflow uses 3.10. This is a known gap — see KNOWN_LIMITATIONS.md.

For Engineers

Architecture, protocol design, model registry, BitTorrent-inspired inference routing, and credit economy mechanics: PROJECT_OVERVIEW.md.

Open limitations (P0–P3 prioritized): KNOWN_LIMITATIONS.md.

Deeper component documentation:

• docs/architecture/ — protocol design and architectural notes
• docs/api/reference.md — HTTP endpoint reference
• docs/deployment/ — Docker / multi-node deployment
• docs/guides/ — development and getting-started guides
• docs/testing/ — test plan and manual testing guide
• bittorrent/README.md — internal docs for the inference-routing component
• MODELS.md — full model catalog (30+ entries) with hardware requirements
• QUICKSTART.md — single-node walkthrough and Docker profile reference

Contributing

This is a closed-source project; contributions are not accepted via public PRs. Internal contributors: see docs/guides/development.md.

Further Reading

Yaarbal originated as research on P2P inference. See docs/research/ for the original protocol design and rebuttal exchange.

License

Copyright 2026 Abdul Hannan Kanji. Licensed under Apache-2.0.