generic-ml-cache-daemon 0.29.0

Local HTTP daemon for generic-ml-cache: REST API, gateway proxy, and session transport. A thin inbound driver over generic-ml-cache-core.

generic-ml-cache-daemon

The HTTP daemon for generic-ml-cache. Exposes the cache store and all session/execution functionality as a local REST API with server-sent event (SSE) streaming, plus a caching proxy gateway for the Anthropic Messages API.

Installation

pip install generic-ml-cache-daemon           # runtime only
pip install "generic-ml-cache-daemon[metrics]" # + Prometheus /metrics endpoint

Starting the daemon

python -m generic_ml_cache_daemon             # uses defaults
GMLCACHE_STORE=/path/to/store python -m generic_ml_cache_daemon
GMLCACHE_SESSION=abc GMLCACHE_METRICS=1 python -m generic_ml_cache_daemon

Environment variables:

Variable	Default	Description
GMLCACHE_STORE	config store path	Path to the cache store directory
GMLCACHE_SESSION	(none)	Bind all intercepted calls to a session
GMLCACHE_METRICS	0	Set 1 to enable the Prometheus /metrics endpoint

HTTP API

The daemon listens on http://127.0.0.1:8765 by default. Interactive API docs are available at /docs (Swagger UI) and /redoc.

Observability

Method	Path	Description
GET	/health	Liveness: {"status":"ok"}
GET	/ready	Readiness: probes the store; 503 if inaccessible
GET	/info	Version, store path, adapters, bound session
GET	/metrics	Prometheus text (requires [metrics] extra + --metrics)

Sessions

Method	Path	Description
GET	/sessions	List all session IDs
POST	/sessions	Create a session (body: {tags, spec})
GET	/sessions/{id}	Get session tags and spec (404 if unknown)
GET	/sessions/{id}/stats	Calls, hits, hit rate
PUT	/sessions/{id}/spec	Set or replace execution spec
DELETE	/sessions/{id}/spec	Remove execution spec
POST	/sessions/{id}/tags	Add a tag
DELETE	/sessions/{id}/tags/{tag}	Remove a tag

Executions & Global Stats

Method	Path	Description
GET	/executions	List all current (servable) executions
GET	/executions/{key}	Inspect by exact key or prefix (409 on ambiguous prefix)
GET	/stats	Global execution count + event counts
POST	/purge	Purge by scope: all, key, tag, session, session_tag

Purge body examples:

{"by": "all"}
{"by": "key", "target": "deadbeef"}
{"by": "session", "target": "abc123"}

Run (synchronous or SSE)

POST /run
{
  "client": "anthropic",
  "model": "claude-opus-4-8",
  "prompt": "Summarise the paper.",
  "effort": "medium",
  "session_id": "abc"
}

• Accept: application/json (default) — blocks and returns {execution_key, state, cache_hit, stdout, stderr}
• Accept: text/event-stream — SSE: {"type":"accepted"} immediately, then {"type":"complete", ...} on finish

Jobs (detached / async)

Method	Path	Description
POST	/jobs	Submit a background execution; returns {job_id, state} with 202
GET	/jobs	List all job IDs
GET	/jobs/{id}	Poll state: pending, running, done, error
GET	/jobs/{id}/stream	SSE: periodic status events, then complete or error

Claude Gateway

POST /gateway/claude/{session_id}/v1/messages

A cache-transparent proxy for the Anthropic Messages API. Requests that hit the cache are returned without a network call to Anthropic. The response is the upstream Anthropic Messages API response returned verbatim — a cache hit is byte-for-byte identical to a live call, with no added fields or headers (the passthrough body must stay exact for SDK compatibility).

⚠️ Security — single-principal, localhost only. The gateway is a single-user tool. Its passthrough cache is keyed on the request body only — deliberately not on the caller's auth token or session (the token refreshes, and keying on it would defeat cross-session cache hits). That is safe for one operator on one machine, but it means two different callers who send the same body share one cache entry regardless of their credentials. Never expose this daemon to a network or to multiple users — a second caller could be served the first caller's provider-authorized response. It binds 127.0.0.1 by default, and a non-loopback --host is refused (a ConfigError) unless you pass --unsafe-allow-network-gateway to loudly accept the risk.

Limitations: single-turn conversations only (one role: user message, no prior assistant turns). Multi-turn support is planned.

Example:

curl http://127.0.0.1:8765/gateway/claude/my-session/v1/messages \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-opus-4-8",
    "messages": [{"role": "user", "content": "Hello, world!"}],
    "max_tokens": 256
  }'

Point any Anthropic SDK client at the gateway by overriding the base URL:

import anthropic

client = anthropic.Anthropic(
    api_key="...",
    base_url="http://127.0.0.1:8765/gateway/claude/my-session",
)

Architecture

The daemon is a thin FastAPI layer over the generic-ml-cache-core hexagonal architecture. It does not own any state — all persistence goes through the existing JournalMetrics (SQLite registry) and ExecutionRepository that the core library manages.

Background jobs run in a ThreadPoolExecutor inside an in-process JobRegistry; job state is not persisted across daemon restarts.

License

Apache-2.0