EggPool

A lightweight, LAN-hosted proxy that aggregates multiple AI provider accounts behind one OpenAI/Anthropic-compatible endpoint.

Features

• Proxies model requests across multiple providers and accounts behind a single endpoint
• OpenAI- and Anthropic-compatible upstream request paths, with transparent bidirectional protocol transcoding
• Dynamically discovers available models; routes by quota utilization (load-based, never cost-based)
• Per-account outbound proxy support (pproxy — SOCKS5, HTTP, Shadowsocks)
• Tracks requests, tokens, latency, errors, and cost provenance in SQLite (provider_reported, trusted local derived/partial, bounded estimated; reservation is advisory, not a floor)
• Multi-page dashboard with 50+ themes, reliability, routing, and runtime views
• Model metadata enrichment from provider catalogs, OpenRouter, Artificial Analysis, and Hugging Face
• Provider-neutral request shaping: cache reporting, safe suffix compression, policy-scoped overrides, optional synthetic cache controls, and advisory threshold tuning
• Thinking/reasoning capability-aware routing with configurable budget mapping
• High-concurrency stream stability: bounded retry queue, lock-contention diagnostics, and an OpenCode-specific operator playbook for sustained coding-agent streaming loads
• Designed for lightweight deployments (Raspberry Pi, SBCs)

Quick Start

# Install (one-shot)
curl -fsSL https://raw.githubusercontent.com/eggstack/eggpool/main/scripts/install.sh | bash

# Interactive onboarding — connect providers, validate, start
eggpool onboard

# Install as a systemd service
sudo env "PATH=$PATH" "$(command -v eggpool)" deploy systemd --install

See Deployment for alternative install methods (pipx, manual, production) and the full deployment guide.

CLI Reference

All commands accept --config /path/to/config.toml. Config resolution: --config > $EGGPOOL_CONFIG > ~/.config/eggpool/config.toml > ./config.toml.

Full command reference: docs/deployment.md

Configuration

Configuration lives in a single TOML file. API keys are loaded from environment variables or .env.

# Example provider configuration
[providers.opencode-go]
id = "opencode-go"
base_url = "https://opencode.ai/zen/go/v1"
protocols = ["openai", "anthropic"]

[[providers.opencode-go.accounts]]
name = "personal"
api_key = "sk-your-opencode-go-key"

Use eggpool connect for interactive provider setup. See docs/providers.md for the full provider catalog, configuration details, and troubleshooting.

Key Config Sections

The catalog refresh is non-destructive by default: failed, empty, or partial upstream responses never silently de-pool a healthy account. Set [models].catalog_withdrawal_policy (preserve_until_health default, confirmed_once, confirmed_twice) to opt into destructive behavior on authoritative refreshes. See architecture/README.md § Catalog Refresh Semantics.

Full config reference: config.example.toml | docs/providers.md

Protocol transcoding

When [transcoder] enabled = true, EggPool bridges OpenAI Chat Completions and Anthropic Messages bidirectionally so a single client ecosystem (e.g. OpenCode, which speaks only OpenAI) can reach Anthropic-only upstreams and vice versa.

What gets translated:

• Request bodies (text + tool-use + vision + thinking + structured outputs)
• Streaming SSE events (including tool-call deltas and thinking deltas)
• Non-retryable error envelopes
• Usage and cost fields (preserved exactly as the upstream reported them)

What is dropped with a structured warning log:

• OpenAI fields with no Anthropic equivalent (logit_bias, presence_penalty, top_logprobs, etc.)
• Anthropic fields with no OpenAI equivalent (top_k, cache_control)

Feature flags ([transcoder.features]) — all off by default:

• tools — bidirectional tool calling translation
• vision — image/document content parts
• thinking — extended thinking ↔ reasoning_content
• structured_outputs — response_format / json_schema coercion
• anthropic_primitives — top_k, cache_control, context_management, container, mcp_servers

The streaming hot path is optimised for sustained concurrent coding-agent loads. See docs/transcoding.md for the full translation table, known limitations, and streaming performance notes.

Request shaping

EggPool includes an opt-in, cache-preserving request-shaping stack. With shipped defaults the entire stack is reporting-only: no request body, header, or route is altered. Routing remains load-based — cache, compression, synthetic-cache, and tuning fields never enter QuotaFairScorer.

The stack covers provider cache counters, request segmentation, native cache preservation, optional compression (observe → safe), optional synthetic cache annotations, and advisory tuning. The dashboard /cache page provides operator summary cards and drill-down tables; advanced diagnostics stay collapsed unless a warning is present.

API Endpoints

When [dashboard].enabled = true, a multi-page dashboard is served at / with request stats, latency metrics, provider health, model-info detail pages, and more. Stats API available under /api/stats/*.

Model-info observability

The dashboard /models page shows enriched model metadata from provider catalogs, OpenRouter, Artificial Analysis, and Hugging Face. It surfaces degraded-state notices when the model-info service is unavailable, and join-failure diagnostics when catalog rows don't match canonical lookups. The /api/stats/runtime endpoint includes a model_info section with source health. See docs/model-info-openrouter-debug.md for the live verification flow.

Documentation

Development

uv sync --extra dev      # install dependencies
uv run pytest            # run all tests
uv run ruff format --check src/ tests/ scripts/
uv run ruff check src/ tests/ scripts/
uv run pyright src/ scripts/

# Optional `orjson` backend for the JSON helper (transcoding hot paths)
uv sync --extra fast     # or: uv pip install 'eggpool[fast]'

# High-concurrency streaming reproducer (no real providers needed)
uv run python scripts/repro_high_concurrency_streams.py --concurrency 50 --cancel-rate 0.25

See AGENTS.md for focused test subset commands.

Agent Configuration

eggpool configsetup generates configuration snippets for popular coding agents:

Shared options: --host, --base-url, --model, --write, --output, --force, --no-clipboard, --print-secret. Generated JSON, TOML, YAML, and shell snippets escape catalog/config values for the target format, including provider-suffixed model IDs.

Examples:

eggpool configsetup aider --model openai/gpt-4 --write
eggpool configsetup continue --model claude-sonnet-4 --output ~/.continue/eggpool.yaml
eggpool configsetup cline --no-clipboard

License

MIT