EggPool

A lightweight, LAN-hosted proxy that aggregates multiple AI provider accounts behind one OpenAI/Anthropic-compatible endpoint.

Features

• Proxies model requests across multiple providers and accounts behind a single endpoint
• OpenAI- and Anthropic-compatible upstream request paths, with transparent bidirectional protocol transcoding
• Dynamically discovers available models; routes by quota utilization (load-based, never cost-based)
• Per-account outbound proxy support (pproxy — SOCKS5, HTTP, Shadowsocks)
• Tracks requests, tokens, latency, errors, and cost provenance in SQLite (provider_reported, trusted local derived/partial, bounded estimated; reservation is advisory, not a floor)
• Multi-page dashboard with 50+ themes, reliability, routing, and runtime views
• Model metadata enrichment from provider catalogs, OpenRouter, Artificial Analysis, and Hugging Face
• Provider-neutral request shaping: cache reporting, safe suffix compression, policy-scoped overrides, optional synthetic cache controls, and advisory threshold tuning
• Thinking/reasoning capability-aware routing with configurable budget mapping
• Designed for lightweight deployments (Raspberry Pi, SBCs)

Quick Start

# Install (one-shot)
curl -fsSL https://raw.githubusercontent.com/eggstack/eggpool/main/scripts/install.sh | bash

# Interactive onboarding — connect providers, validate, start
eggpool onboard

# Install as a systemd service
sudo env "PATH=$PATH" "$(command -v eggpool)" deploy systemd --install

See Deployment for alternative install methods (pipx, manual, production) and the full deployment guide.

CLI Reference

All commands accept --config /path/to/config.toml. Config resolution: --config > $EGGPOOL_CONFIG > ~/.config/eggpool/config.toml > ./config.toml.

Full command reference: docs/deployment.md

Configuration

Configuration lives in a single TOML file. API keys are loaded from environment variables or .env.

# Example provider configuration
[providers.opencode-go]
id = "opencode-go"
base_url = "https://opencode.ai/zen/go/v1"
protocols = ["openai", "anthropic"]

[[providers.opencode-go.accounts]]
name = "personal"
api_key = "sk-your-opencode-go-key"

Use eggpool connect for interactive provider setup. See docs/providers.md for the full provider catalog, configuration details, and troubleshooting.

Key Config Sections

The catalog refresh is non-destructive by default: failed, empty, or partial upstream responses never silently de-pool a healthy account. Set [models].catalog_withdrawal_policy (preserve_until_health default, confirmed_once, confirmed_twice) to opt into destructive behavior on authoritative refreshes. See architecture/README.md § Catalog Refresh Semantics.

Full config reference: config.example.toml | docs/providers.md

Protocol transcoding

When [transcoder] enabled = true, EggPool bridges OpenAI Chat Completions and Anthropic Messages bidirectionally so a single client ecosystem (e.g. OpenCode, which speaks only OpenAI) can reach Anthropic-only upstreams and vice versa.

What gets translated:

• Request bodies (text + tool-use + vision + thinking + structured outputs)
• Streaming SSE events (including tool-call deltas and thinking deltas)
• Non-retryable error envelopes
• Usage and cost fields (preserved exactly as the upstream reported them)

What is dropped with a structured warning log:

• OpenAI fields with no Anthropic equivalent (logit_bias, presence_penalty, top_logprobs, etc.)
• Anthropic fields with no OpenAI equivalent (top_k, cache_control)

Phase 6 feature flags ([transcoder.features]) — all off by default:

• tools — bidirectional tool calling translation
• vision — image/document content parts
• thinking — extended thinking ↔ reasoning_content
• structured_outputs — response_format / json_schema coercion
• anthropic_primitives — top_k, cache_control, context_management, container, mcp_servers

See docs/transcoding.md for the full translation table and known limitations.

Request shaping

EggPool’s request-shaping stack is opt-in and cache-preserving by default. With the shipped config, the public surface stays in reporting mode: no route changes, no stable-prefix mutation, and no synthetic cache annotations on the wire. Routing remains load-based; cache, compression, synthetic-cache, and tuning fields never enter QuotaFairScorer.

The detailed request-shaping summary and drill-down cards live on /cache; /runtime keeps a compact relocation panel that points back to that page.

Operator surfaces

Stable config knobs

The default example now exposes only the normal operator knobs:

[compression]
enabled = false
mode = "observe"
min_candidate_tokens = 2048
min_savings_tokens = 1024
max_compression_latency_ms = 25.0

[compression.transforms]
fold_repeated_lines = true
compact_logs = true
compact_search_results = true
elide_base64_blobs = true
minify_machine_json = true
compact_stack_traces = true

[cache.synthetic_cache_controls]
enabled = false
dry_run = true
min_stable_tokens = 1024

Advanced knobs still exist for policy-scoped rollouts and advisory tuning, but they are intentionally pushed into the docs instead of the main example:

• [[compression.policies]] for scoped overrides
• compression.placement, respect_cache_boundaries, header_* for advanced routing-adjacent operator workflows
• cache.synthetic_cache_controls.provider_kinds, ttl, max_breakpoints, placements, require_policy
• compression.tuning.* for recommendation-only threshold guidance

Safety rules

• Safe compression mutates only eligible volatile_suffix string leaves.
• Stable-prefix preservation is verified with stable_prefix_content_hash; any mismatch falls back to the original payload.
• Native provider cache annotations are preserved byte-for-byte.
• Synthetic cache controls are disabled by default and dry-run first when enabled.
• Context-limit checks happen before compression; compression never rescues an over-limit request.
• Routing stays load-based and reporting-only metrics never influence scorer inputs.

Recommended rollout

1. Enable cache reporting and request segmentation only.
2. Run compression in observe mode for 24-48 hours.
3. Turn on safe compression for one client or policy.
4. Expand safe compression if fallbacks stay at zero.
5. Try synthetic cache controls in dry-run for Anthropic-compatible upstreams.
6. Move synthetic cache to apply mode only after the dry-run is clean.
7. Enable advisory tuning only if you want threshold suggestions.

Further reading

• Operator guide: docs/cache-compression.md
• Copy-pasteable profiles: docs/cache-compression-profiles.md
• Troubleshooting: docs/cache-compression-troubleshooting.md
• Architecture summary: architecture/README.md

API Endpoints

When [dashboard].enabled = true, a multi-page dashboard is served at / with request stats, latency metrics, provider health, model-info detail pages, and more. Stats API available under /api/stats/*.

Model-info observability

• The dashboard /models page renders a degraded-state notice above the table if the model-info service is unattached (no app.state.model_info) or if get_summary_map() raises an exception. The exception's full traceback is logged under eggpool.dashboard.routes — the page never embeds the traceback text in HTML.
• The dashboard /models/{model_id:path} detail page distinguishes "no canonical row exists" (empty-state copy) from "lookup failed" (degraded-state notice) when the service throws.
• /api/stats/runtime includes a model_info section with enabled, canonical_count, catalog_model_count, provider_model_count, due_count, and a source_health dict (no raw source payloads). Failures surface as *_error keys and probe_errors, never as raised exceptions.
• OpenRouter source health reflects catalog availability, not local match success — a successful fetch with zero matches still updates last_success_at / last_payload_count. When a forced refresh finds configured aliases but no catalog match, the OpenRouter cache is invalidated and the fetch retried once (recorded as cache_retry: true under source_diagnostics.openrouter).
• GET /api/model-info/{model_id} returns observations rows read from model_info_observations (per source: source_model_id, provider_id, observed_at, confidence, display name, context window, modalities). Raw payloads are never returned. When no rows exist for a canonical model, the field is a synthetic placeholder flagged _synthetic: true so callers can distinguish real observation data from fallback synthesis. When the repository read fails the response returns observations: [] plus an observations_error: <ExcClass> key instead of synthesising rows — operators see "no data" rather than fabricated external source ids.
• The dashboard model detail page mirrors those observation rows in an Observations panel. External display_name_<source> values (e.g. MiniMax: MiniMax M3) are promoted into detail.display_name only when the provider did not seed one; detail.display_name_source records the chosen source. Source-scoped advisory pricing (e.g. OpenRouter's $/Mtok) lives under detail.pricing.<source>, separate from authoritative local cost accounting. When the observation read fails the dashboard renders an "Observation read failed" panel with the error class name.
• Alias and observation lookups are case-insensitive at the repository layer (lower(model_id) = lower(?)), so provider casing drift (MiniMax-M3 vs minimax-m3) does not break refresh or detail lookup. Manual refreshes reseed configured [model_info.aliases] before external matching, so newly added aliases apply without a process restart. Alias candidate selection is deterministic: exact-case rows win over case-folded rows; identical alias strings are deduplicated; conflicting folded aliases with no exact-case match produce an unambiguous no-match. The resolver exposes source_diagnostics.openrouter.alias_rows (one entry per candidate with match_kind = "exact_case" | "case_folded") and alias_selection so operators can audit the choice.
• scripts/debug_model_info_openrouter.sh runs the live verification flow (force refresh → detail GET → source-health query) for any model id; the expected outcomes are documented in docs/model-info-openrouter-debug.md.

Documentation

Development

uv sync --extra dev      # install dependencies
uv run pytest            # run all tests
uv run ruff format --check src/ tests/ scripts/
uv run ruff check src/ tests/ scripts/
uv run pyright src/ scripts/

See AGENTS.md for focused test subset commands.

Agent Configuration

eggpool configsetup generates configuration snippets for popular coding agents:

Shared options: --host, --base-url, --model, --write, --output, --force, --no-clipboard, --print-secret. Generated JSON, TOML, YAML, and shell snippets escape catalog/config values for the target format, including provider-suffixed model IDs.

Examples:

eggpool configsetup aider --model openai/gpt-4 --write
eggpool configsetup continue --model claude-sonnet-4 --output ~/.continue/eggpool.yaml
eggpool configsetup cline --no-clipboard

License

MIT