EggPool

A lightweight, LAN-hosted proxy that aggregates multiple AI provider accounts behind one OpenAI/Anthropic-compatible endpoint.

Features

• Proxies model requests across multiple providers and accounts behind a single endpoint
• OpenAI- and Anthropic-compatible upstream request paths, with transparent bidirectional protocol transcoding
• Dynamically discovers available models; routes by quota utilization (load-based, never cost-based)
• Per-account outbound proxy support (pproxy — SOCKS5, HTTP, Shadowsocks)
• Tracks requests, tokens, latency, errors, and cost provenance in SQLite (provider_reported, trusted local derived/partial, bounded estimated)
• Multi-page dashboard with 50+ themes, reliability, routing, and runtime views
• Model metadata enrichment from provider catalogs, OpenRouter, Artificial Analysis, and Hugging Face
• Provider-neutral cache observability and a cache-preserving compression stack (segmentation, transcoder cache stability, observe/safe suffix compression, per-policy overrides, optional synthetic cache controls, closed-loop threshold tuning)
• Thinking/reasoning capability-aware routing with configurable budget mapping
• Designed for lightweight deployments (Raspberry Pi, SBCs)

Quick Start

# Install (one-shot)
curl -fsSL https://raw.githubusercontent.com/eggstack/eggpool/main/scripts/install.sh | bash

# Interactive onboarding — connect providers, validate, start
eggpool onboard

# Install as a systemd service
sudo env "PATH=$PATH" "$(command -v eggpool)" deploy systemd --install

See Deployment for alternative install methods (pipx, manual, production) and the full deployment guide.

CLI Reference

All commands accept --config /path/to/config.toml. Config resolution: --config > $EGGPOOL_CONFIG > ~/.config/eggpool/config.toml > ./config.toml.

Full command reference: docs/deployment.md

Configuration

Configuration lives in a single TOML file. API keys are loaded from environment variables or .env.

# Example provider configuration
[providers.opencode-go]
id = "opencode-go"
base_url = "https://opencode.ai/zen/go/v1"
protocols = ["openai", "anthropic"]

[[providers.opencode-go.accounts]]
name = "personal"
api_key = "sk-your-opencode-go-key"

Use eggpool connect for interactive provider setup. See docs/providers.md for the full provider catalog, configuration details, and troubleshooting.

Key Config Sections

The catalog refresh is non-destructive by default: failed, empty, or partial upstream responses never silently de-pool a healthy account. Set [models].catalog_withdrawal_policy (preserve_until_health default, confirmed_once, confirmed_twice) to opt into destructive behavior on authoritative refreshes. See architecture/README.md § Catalog Refresh Semantics.

Full config reference: config.example.toml | docs/providers.md

Protocol transcoding

When [transcoder] enabled = true, EggPool bridges OpenAI Chat Completions and Anthropic Messages bidirectionally so a single client ecosystem (e.g. OpenCode, which speaks only OpenAI) can reach Anthropic-only upstreams and vice versa.

What gets translated:

• Request bodies (text + tool-use + vision + thinking + structured outputs)
• Streaming SSE events (including tool-call deltas and thinking deltas)
• Non-retryable error envelopes
• Usage and cost fields (preserved exactly as the upstream reported them)

What is dropped with a structured warning log:

• OpenAI fields with no Anthropic equivalent (logit_bias, presence_penalty, top_logprobs, etc.)
• Anthropic fields with no OpenAI equivalent (top_k, cache_control)

Phase 6 feature flags ([transcoder.features]) — all off by default:

• tools — bidirectional tool calling translation
• vision — image/document content parts
• thinking — extended thinking ↔ reasoning_content
• structured_outputs — response_format / json_schema coercion
• anthropic_primitives — top_k, cache_control, context_management, container, mcp_servers

See docs/transcoding.md for the full translation table and known limitations.

Cache observability & safe suffix compression

EggPool ships a cache-preserving compression stack that is observational and routing-isolated by default. Routing is load-based (request count + token count + active count + health); no cache, compression, synthetic-cache, or tuning field is ever consumed by QuotaFairScorer. A hardcoded runtime diagnostic (GET /api/stats/runtime.guardrails) and tests/unit/test_routing_guardrails.py pin the invariant.

The stack exposes the following surfaces:

• Cache observability — every finalized request records a cache_counter_status of reported / not_reported / unknown_format plus parsed cache-token counts. Dashboard: "Runtime → Cache observability"; JSON: GET /api/stats/cache-observability.
• Canonical request segmentation — every finalized request is annotated into stable_prefix / semi_stable_context / volatile_suffix regions without mutating the payload, with concrete JSON content_path values that resolve to actual string leaves. Dashboard: "Runtime → Segmentation"; JSON: GET /api/stats/canonical-request-segmentation.
• Transcoder cache stability — cross-protocol requests carry a cache_boundary_tracker recording whether cache_control annotations were preserved, relocated, or dropped, plus a deterministic SHA-256 of the provider-visible stable prefix. JSON: GET /api/stats/cache-stability.
• Safe suffix compression — when [compression] mode = "safe", six deterministic transforms (fold_repeated_lines, compact_logs, compact_search_results, compact_stack_traces, elide_base64_blobs, minify_machine_json) fold eligible volatile_suffix content. stable_prefix is recomputed SHA-256 verified after every transform; any mismatch degrades to the original payload with failed_fallback=True. Default mode is observe (reporting only). Dashboard: "Runtime → Compression"; JSON: GET /api/stats/compression-observability, /api/stats/compression-runtime.
• Per-policy overrides — [[compression.policies]] rows target clients, protocols, models, or transcoding paths without changing the global config. Dashboard: "Runtime → Compression policy"; JSON: GET /api/stats/compression-policies.
• Synthetic cache controls — opt-in post-route cache_control annotations for providers that support explicit cache boundaries (Anthropic-style). Disabled by default; dry-run by default. Stable-prefix only; native cache_control preserved byte-for-byte; only ttl = "ephemeral" is currently accepted.
• Closed-loop threshold tuning — advisory recommendations for min_candidate_tokens, min_savings_tokens, max_compression_latency_ms. mode = "recommend" writes recommendations; mode = "apply" is accepted at config but does not currently register runtime overrides.

Headers: x-eggpool-compression: off|observe|safe (when header_override = true) and x-eggpool-cache-policy: preserve to opt out per request.

Privacy invariants: no raw prompt, tool output, system message, request body, auth header, or provider API key is ever shown or persisted in any cache, compression, or synthetic-cache surface.

Full operator guide: docs/cache-compression.md. Copy-pasteable profiles: docs/cache-compression-profiles.md. Symptom-to-cause troubleshooting: docs/cache-compression-troubleshooting.md.

API Endpoints

When [dashboard].enabled = true, a multi-page dashboard is served at / with request stats, latency metrics, provider health, model-info detail pages, and more. Stats API available under /api/stats/*.

Model-info observability

• The dashboard /models page renders a degraded-state notice above the table if the model-info service is unattached (no app.state.model_info) or if get_summary_map() raises an exception. The exception's full traceback is logged under eggpool.dashboard.routes — the page never embeds the traceback text in HTML.
• The dashboard /models/{model_id:path} detail page distinguishes "no canonical row exists" (empty-state copy) from "lookup failed" (degraded-state notice) when the service throws.
• /api/stats/runtime includes a model_info section with enabled, canonical_count, catalog_model_count, provider_model_count, due_count, and a source_health dict (no raw source payloads). Failures surface as *_error keys and probe_errors, never as raised exceptions.

Documentation

Development

uv sync --extra dev
uv run ruff check src/ tests/ scripts/
uv run ruff format src/ tests/ scripts/
uv run pyright src/ scripts/
uv run pytest

Agent Configuration

eggpool configsetup generates configuration snippets for popular coding agents:

Shared options: --host, --base-url, --model, --write, --output, --force, --no-clipboard, --print-secret. Generated JSON, TOML, YAML, and shell snippets escape catalog/config values for the target format, including provider-suffixed model IDs.

Examples:

eggpool configsetup aider --model openai/gpt-4 --write
eggpool configsetup continue --model claude-sonnet-4 --output ~/.continue/eggpool.yaml
eggpool configsetup cline --no-clipboard

License

MIT