EggPool

A lightweight, LAN-hosted proxy that aggregates multiple AI provider accounts behind one OpenAI/Anthropic-compatible endpoint.

Features

• Proxies model requests across multiple providers and accounts behind a single endpoint
• OpenAI- and Anthropic-compatible upstream request paths, with transparent bidirectional protocol transcoding
• Dynamically discovers available models; routes by quota utilization (load-based, never cost-based)
• Per-account outbound proxy support (pproxy — SOCKS5, HTTP, Shadowsocks)
• Tracks requests, tokens, latency, errors, and cost provenance in SQLite (provider_reported, trusted local derived/partial, bounded estimated; reservation is advisory, not a floor)
• Multi-page dashboard with 50+ themes, reliability, routing, and runtime views
• Model metadata enrichment from provider catalogs, OpenRouter, Artificial Analysis, and Hugging Face
• Provider-neutral request shaping: cache reporting, safe suffix compression, policy-scoped overrides, optional synthetic cache controls, and advisory threshold tuning
• Thinking/reasoning capability-aware routing with configurable budget mapping
• High-concurrency stream stability: bounded retry queue, lock-contention diagnostics, and an OpenCode-specific operator playbook for sustained coding-agent streaming loads
• Designed for lightweight deployments (Raspberry Pi, SBCs)

Quick Start

# Install (one-shot)
curl -fsSL https://raw.githubusercontent.com/eggstack/eggpool/main/scripts/install.sh | bash

# Interactive onboarding — connect providers, validate, start
eggpool onboard

# Install as a systemd service
sudo env "PATH=$PATH" "$(command -v eggpool)" deploy systemd --install

See Deployment for alternative install methods (pipx, manual, production) and the full deployment guide.

CLI Reference

All commands accept --config /path/to/config.toml. Config resolution: --config > $EGGPOOL_CONFIG > ~/.config/eggpool/config.toml > ./config.toml.

Full command reference: docs/deployment.md

Configuration

Configuration lives in a single TOML file. API keys are loaded from environment variables or .env.

# Example provider configuration
[providers.opencode-go]
id = "opencode-go"
base_url = "https://opencode.ai/zen/go/v1"
protocols = ["openai", "anthropic"]

[[providers.opencode-go.accounts]]
name = "personal"
api_key = "sk-your-opencode-go-key"

Use eggpool connect for interactive provider setup. See docs/providers.md for the full provider catalog, configuration details, and troubleshooting.

Key Config Sections

The catalog refresh is non-destructive by default: failed, empty, or partial upstream responses never silently de-pool a healthy account. Set [models].catalog_withdrawal_policy (preserve_until_health default, confirmed_once, confirmed_twice) to opt into destructive behavior on authoritative refreshes. See architecture/README.md § Catalog Refresh Semantics.

Full config reference: config.example.toml | docs/providers.md

Protocol transcoding

When [transcoder] enabled = true, EggPool bridges OpenAI Chat Completions and Anthropic Messages bidirectionally so a single client ecosystem (e.g. OpenCode, which speaks only OpenAI) can reach Anthropic-only upstreams and vice versa.

What gets translated:

• Request bodies (text + tool-use + vision + thinking + structured outputs)
• Streaming SSE events (including tool-call deltas and thinking deltas)
• Non-retryable error envelopes
• Usage and cost fields (preserved exactly as the upstream reported them)

What is dropped with a structured warning log:

• OpenAI fields with no Anthropic equivalent (logit_bias, presence_penalty, top_logprobs, etc.)
• Anthropic fields with no OpenAI equivalent (top_k, cache_control)

Phase 6 feature flags ([transcoder.features]) — all off by default:

• tools — bidirectional tool calling translation
• vision — image/document content parts
• thinking — extended thinking ↔ reasoning_content
• structured_outputs — response_format / json_schema coercion
• anthropic_primitives — top_k, cache_control, context_management, container, mcp_servers

The streaming hot path is optimised for sustained concurrent coding-agent loads: only the coordinator runs an SSE observer (the transcoder no longer runs its own for usage extraction), StreamingTranscoder.feed()/.flush() are synchronous, translated output per upstream chunk is coalesced into a single yield, and frame helpers use compact JSON separators. See docs/transcoding.md for the full translation table, known limitations, and the streaming hot-path notes.

Request shaping

EggPool’s request-shaping stack is opt-in and cache-preserving by default. With the shipped config, the public surface stays in reporting mode: no route changes, no stable-prefix mutation, and no synthetic cache annotations on the wire. Routing remains load-based; cache, compression, synthetic-cache, and tuning fields never enter QuotaFairScorer.

The detailed request-shaping summary and drill-down cards live on /cache; /runtime keeps a compact relocation panel that points back to that page.

The /cache page opens with six top-level operator summary cards:

Raw reporting_only and other internal modes survive in the subtext, not the primary metric. Advanced diagnostics stay collapsed unless a warning or non-default state is present (segmentation parse failures, synthetic-cache warnings or applied count, tuning recommendations, routing guardrail violation, transcoding loss warnings, compression warnings or policy warnings).

Operator surfaces

Stable config knobs

The default example now exposes only the normal operator knobs:

[compression]
enabled = false
mode = "observe"
min_candidate_tokens = 2048
min_savings_tokens = 1024
max_compression_latency_ms = 25.0

[compression.transforms]
fold_repeated_lines = true
compact_logs = true
compact_search_results = true
elide_base64_blobs = true
minify_machine_json = true
compact_stack_traces = true

[cache.synthetic_cache_controls]
enabled = false
dry_run = true
min_stable_tokens = 1024

Advanced knobs still exist for policy-scoped rollouts and advisory tuning, but they are intentionally pushed into the docs instead of the main example:

• [[compression.policies]] for scoped overrides
• compression.placement, respect_cache_boundaries, header_* for advanced routing-adjacent operator workflows
• cache.synthetic_cache_controls.provider_kinds, ttl, max_breakpoints, placements, require_policy
• compression.tuning.* for recommendation-only threshold guidance

Safety rules

• Safe compression mutates only eligible volatile_suffix string leaves; no-op runs return the original payload by identity and applied runs use path-level copy-on-write (not a deep copy).
• Stable-prefix preservation is verified with stable_prefix_content_hash; any mismatch falls back to the original payload.
• Native provider cache annotations are preserved byte-for-byte.
• Synthetic cache controls are disabled by default and dry-run first when enabled.
• Context-limit checks happen before compression; compression never rescues an over-limit request.
• Routing stays load-based and reporting-only metrics never influence scorer inputs.

Recommended rollout

1. Enable cache reporting and request segmentation only.
2. Run compression in observe mode for 24-48 hours.
3. Turn on safe compression for one client or policy.
4. Expand safe compression if fallbacks stay at zero.
5. Try synthetic cache controls in dry-run for Anthropic-compatible upstreams.
6. Move synthetic cache to apply mode only after the dry-run is clean.
7. Enable advisory tuning only if you want threshold suggestions.

Further reading

See /cache for the operator workflow: requests being changed, provider cache counters, compression outcomes, safety guardrails, routing isolation. Advanced diagnostics stay collapsed unless warnings are present.

• Operator guide: docs/cache-compression.md
• Copy-pasteable profiles: docs/cache-compression-profiles.md
• Troubleshooting: docs/cache-compression-troubleshooting.md
• Architecture summary: architecture/README.md

API Endpoints

When [dashboard].enabled = true, a multi-page dashboard is served at / with request stats, latency metrics, provider health, model-info detail pages, and more. Stats API available under /api/stats/*.

Model-info observability

• The dashboard /models page renders a degraded-state notice above the table if the model-info service is unattached (no app.state.model_info) or if get_summary_map() raises an exception. The exception's full traceback is logged under eggpool.dashboard.routes — the page never embeds the traceback text in HTML. When the canonical summary map has rows but no rendered dashboard row matches any of them, a separate join-failure diagnostic appears (with up to five unmatched sample rows for grep-based debugging) — this catches the "API correct / dashboard empty" state caused by provider-suffixed ids not being normalized to canonical lookup keys. Catalog row construction failures (get_provider_model_entries, get_models_for_exposure) are logged with traceback and surfaced as degraded_reason="fetch_error" instead of silently dropping the table.
• The provider-scoped catalog accessor (ModelCatalogCache.get_provider_model_entries()) returns a deterministic dict[(model_id, provider_id), dict] view that excludes the deprecated placeholder, applies configured capability overrides when cache._config is attached, and emits shallow copies so mutations cannot leak back into the cache. Unresolved entries (protocol=None) are kept so the dashboard can render them with available=False, catalog_status="unavailable" rather than silently omitting them.
• The dashboard /models/{model_id:path} detail page distinguishes "no canonical row exists" (empty-state copy) from "lookup failed" (degraded-state notice) when the service throws.
• /api/stats/runtime includes a model_info section with enabled, canonical_count, catalog_model_count, provider_model_count, due_count, and a source_health dict (no raw source payloads). Failures surface as *_error keys and probe_errors, never as raised exceptions.
• OpenRouter source health reflects catalog availability, not local match success — a successful fetch with zero matches still updates last_success_at / last_payload_count. When a forced refresh finds configured aliases but no catalog match, the OpenRouter cache is invalidated and the fetch retried once (recorded as cache_retry: true under source_diagnostics.openrouter).
• GET /api/model-info/{model_id} returns observations rows read from model_info_observations (per source: source_model_id, provider_id, observed_at, confidence, display name, context window, modalities). Raw payloads are never returned. When no rows exist for a canonical model, the field is a synthetic placeholder flagged _synthetic: true so callers can distinguish real observation data from fallback synthesis. When the repository read fails the response returns observations: [] plus an observations_error: <ExcClass> key instead of synthesising rows — operators see "no data" rather than fabricated external source ids.
• The dashboard model detail page mirrors those observation rows in an Observations panel. External display_name_<source> values (e.g. MiniMax: MiniMax M3) are promoted into detail.display_name only when the provider did not seed one; detail.display_name_source records the chosen source. Source-scoped advisory pricing (e.g. OpenRouter's $/Mtok) lives under detail.pricing.<source>, separate from authoritative local cost accounting. When the observation read fails the dashboard renders an "Observation read failed" panel with the error class name.
• Alias and observation lookups are case-insensitive at the repository layer (lower(model_id) = lower(?)), so provider casing drift (MiniMax-M3 vs minimax-m3) does not break refresh or detail lookup. Manual refreshes reseed configured [model_info.aliases] before external matching, so newly added aliases apply without a process restart. Alias candidate selection is deterministic: exact-case rows win over case-folded rows; identical alias strings are deduplicated; conflicting folded aliases with no exact-case match produce an unambiguous no-match. The resolver exposes source_diagnostics.openrouter.alias_rows (one entry per candidate with match_kind = "exact_case" | "case_folded") and alias_selection so operators can audit the choice.
• Deployment-suffix matching (tier 2b, opt-in via matching.deployment_suffix_normalized_exact = true) strips deployment-shaped tokens from DEPLOYMENT_SUFFIX_TOKENS (highspeed, fast, turbo, speed, lowlatency, lowlat) only when the original identifier carries a digit/family anchor, the candidate set is unique, and the original contains no semantic-variant token from SEMANTIC_VARIANT_TOKENS (pro, mini, flash, lite, max, plus, instruct, chat, reasoning, thinking, preview, code, coder, omni). So MiniMax-M2.7-highspeed resolves to minimax/minimax-m2.7 while claude-3-haiku-highspeed keeps its semantic shape. Note that preview, max, and similar semantic tokens are deliberately not stripped even though they look like deployment markers. Artificial Analysis matching shares the same tiered resolver and the same match-evidence audit trail as OpenRouter.
• build_canonical_detail() credits provenance.sources for any existing_detail.external_ids[*] key that wasn't contributed this cycle, populating provenance.source_states[<src>] = "preserved_external_id". Detail cycles therefore distinguish three source states explicitly: contributed (newly fetched this cycle), preserved_external_id (carried from the prior canonical row because the external ID persists in external_ids), and absent (no observation and no external ID).
• scripts/debug_model_info_openrouter.sh runs the live verification flow (force refresh → detail GET → source-health query) for any model id; the expected outcomes are documented in docs/model-info-openrouter-debug.md.

Documentation

Development

uv sync --extra dev      # install dependencies
uv run pytest            # run all tests
uv run ruff format --check src/ tests/ scripts/
uv run ruff check src/ tests/ scripts/
uv run pyright src/ scripts/

# Optional `orjson` backend for the JSON helper (transcoding hot paths)
uv sync --extra fast     # or: uv pip install 'eggpool[fast]'

# High-concurrency streaming reproducer (no real providers needed)
uv run python scripts/repro_high_concurrency_streams.py --concurrency 50 --cancel-rate 0.25

See AGENTS.md for focused test subset commands.

Agent Configuration

eggpool configsetup generates configuration snippets for popular coding agents:

Shared options: --host, --base-url, --model, --write, --output, --force, --no-clipboard, --print-secret. Generated JSON, TOML, YAML, and shell snippets escape catalog/config values for the target format, including provider-suffixed model IDs.

Examples:

eggpool configsetup aider --model openai/gpt-4 --write
eggpool configsetup continue --model claude-sonnet-4 --output ~/.continue/eggpool.yaml
eggpool configsetup cline --no-clipboard

License

MIT