mcp-persist 1.12.0

Production-grade persistence backends for the MCP Python SDK

mcp-persist

When an MCP client reconnects, the server has to replay the events it missed, and with only the SDK's in-memory EventStore, that replay is impossible: the session lived in one process's memory, so a restart or a reconnect to a different worker loses it. mcp-persist adds drop-in durable EventStore backends for SQLite, Redis, and PostgreSQL that survive process restarts and scale across multi-worker deployments, keeping SSE stream resumability intact.

📚 This README is the quick tour. Full reference lives in docs/: backends, CLI, the programmatic API, architecture, benchmarks, and the production guide.

Quickstart: with_persistence()

The fastest way to add resumability to a FastMCP server. Wiring it by hand means an event store, a StreamableHTTPSessionManager, a Starlette lifespan to open and close them, and a Mount. with_persistence() collapses all of it to two lines: pass your FastMCP instance and get back a runnable Starlette ASGI app with the store and session manager already wired in, opened on startup and closed on shutdown.

import uvicorn
from mcp.server.fastmcp import FastMCP
from mcp_persist import with_persistence

mcp = FastMCP(name="MyServer")

# Swap backend="redis" / "postgres" with the matching url:
app = with_persistence(mcp, backend="sqlite", url="events.db", ttl=3600)
uvicorn.run(app, host="127.0.0.1", port=8000)  # MCP endpoint at /mcp

That replaces ~35 lines of manual lifespan/Mount/session-manager boilerplate. There are three ways to supply the store, resolved in order:

# A: config kwargs; the app builds the store and owns its lifecycle:
app = with_persistence(mcp, backend="redis", url="redis://localhost:6379", ttl=3600)

# B: bring your own store; you own its lifecycle (the app does NOT close it):
async with SQLiteEventStore.create("events.db", ttl=3600) as store:
    app = with_persistence(mcp, store=store)
    await uvicorn.Server(uvicorn.Config(app, port=8000)).serve()

# C: configure from the environment (MCP_PERSIST_BACKEND / _URL / _TTL / …):
app = with_persistence(mcp)

The live store is exposed on app.state.event_store, so you can run a PurgeScheduler alongside the server. No extra dependency is required: starlette and the session manager ship with mcp. See examples/fastmcp_plugin_server.py.

Under the hood, whichever setup you use, it's the same layering: a StreamableHTTPSessionManager backed by a durable EventStore you choose.

MCP Server
     │
     ▼
StreamableHTTPSessionManager
     │
     ▼
EventStore
 ├─ SQLite
 ├─ Redis
 └─ PostgreSQL

Not on FastMCP, or want to own the wiring yourself? Build a store and pass it to StreamableHTTPSessionManager directly; see Manual wiring.

Resumability without touching the server: PersistenceProxy

When you can't (or don't want to) modify the MCP server, such as a third-party server, another language, or a binary you don't own, run the proxy in front of it. It forwards requests upstream and intercepts the SSE responses, persisting every event to a store and assigning its own event IDs. A client that disconnects reconnects with Last-Event-ID; the proxy replays the missed events from the store and continues live. The upstream needs no event store of its own: the proxy is the store.

Running a TypeScript (or any non-Python) MCP server? The proxy speaks plain HTTP, so it adds resumability in front of it without touching the server. See docs/typescript.md for a step-by-step guide.

Point your clients at the proxy's address instead of the server's (e.g. http://localhost:8000/mcp); nothing else on the client changes. Resumability rides the standard SSE Last-Event-ID header, so any MCP client that reconnects after a drop gets its missed events back automatically.

# Point at a running MCP server (no extra install needed: httpx & uvicorn
# already ship with mcp):
mcp-persist-proxy --upstream http://localhost:8001 \
    --backend sqlite --url events.db --port 8000

# …or start the server as a subprocess, wait for it, and proxy it:
mcp-persist-proxy --backend redis --url redis://localhost:6379 \
    --port 8000 --upstream-port 8001 -- uvicorn my_server:app --port 8001

Or embed it as an ASGI app:

import uvicorn
from mcp_persist import PersistenceProxy

async def serve():
    async with PersistenceProxy.create(
        "http://localhost:8001", backend="sqlite", url="events.db", ttl=3600
    ) as proxy:
        await uvicorn.Server(uvicorn.Config(proxy, port=8000)).serve()

The store is resolved exactly like with_persistence: a pre-built store=, backend=+url=, or MCP_PERSIST_* env vars. (ttl is how long stored events are kept, in seconds; it's available as --ttl on the CLI too.)

What it does and does not do. It adds resumability against a stable upstream: a server that stays up while clients come and go. It survives client disconnects (flaky networks, mobile, tunnels), and, with a durable store like SQLite or Postgres, a restart of the proxy itself. Two things it can't do: it can't recover from the upstream server restarting: a restarted server is a clean break, so the proxy can replay what it already stored but can't carry the old connection over to the new server; and it can't replay an event that was never stored: if the client and the proxy both drop before an event is saved, it's gone. It never makes delivery less reliable than talking to the server directly.

Browser clients (CORS). A browser-based MCP client (a web UI) talks to the proxy through fetch, so it needs CORS. Pass --cors to let the proxy answer the preflight itself and stamp Access-Control-Allow-Origin on every response (including the SSE streams it synthesizes, which is where a browser otherwise fails with "Failed to fetch"). It also exposes mcp-session-id so the client's JavaScript can read the session id. --cors allows any origin (*); pass an explicit origin to restrict it (--cors https://app.example):

mcp-persist-proxy --upstream http://localhost:8001 \
    --backend sqlite --url events.db --port 8000 --cors

Command-line tools

Diagnostic commands for operating a live store, plus an upstream pre-flight for the proxy. The store commands resolve their target from --backend/--url flags or the MCP_PERSIST_* env vars. Full reference, sample output, JSON schema, and exit-code semantics in docs/cli.md.

# Pass/fail health checklist (runtime, driver, connectivity, retention):
mcp-persist doctor --backend sqlite --url events.db --ttl 3600

# Per-stream event inventory + latency probe:
mcp-persist stats --backend sqlite --url events.db

# Force a purge of expired events (--dry-run to count first, --older-than by age):
mcp-persist purge --backend sqlite --url events.db --ttl 3600

# Export one stream to portable JSON and load it into a fresh store:
mcp-persist dump session-abc --backend sqlite --url events.db -o session.json
mcp-persist load session.json --backend sqlite --url repro.db

# Copy every stream from one backend to another:
mcp-persist migrate --from-backend sqlite --from-url events.db \
    --to-backend postgres --to-url postgresql://localhost/app

# Verify an upstream is reachable and speaks Streamable HTTP, then exit:
mcp-persist-proxy --upstream http://localhost:8001 --check

Backends & choosing one

Backend	Extra	Use case
SQLiteEventStore	sqlite	Single-process SSE resumability across restarts, with no external service
RedisEventStore	redis	Multi-process / multi-worker SSE resumability
PostgresEventStore	postgres	Durable resumability for deployments already running Postgres, including multi-node / team setups

Start from how you deploy:

If your deployment…	Use
Runs as a single process and you want zero extra infrastructure	SQLiteEventStore
Runs multiple workers / replicas behind a load balancer	RedisEventStore
Already runs PostgreSQL, or needs durable storage at team / multi-node scale	PostgresEventStore
Runs on serverless / a read-only or ephemeral filesystem	RedisEventStore or PostgresEventStore (never SQLite)

Any replica count > 1 needs a shared store (Redis/Postgres), not SQLite. A local SQLite file is visible only to the process that opened it, so behind a load balancer (or during a rolling deploy, when a reconnecting client lands on a different pod) that pod won't have the client's events and the resume silently returns nothing. SQLite is for a genuine single process. See deployment topologies.

How they compare:

	SQLite	Redis	Postgres
External service	None	Redis	PostgreSQL
Multi-process / multi-worker	No (single writer)	Yes	Yes
Durable across restarts	Yes (on disk)	Depends on Redis persistence config	Yes
Automatic expiry	No (call purge_expired())	Yes (native key TTL)	No (call purge_expired())
Best fit	Single node, edge, local dev	Load-balanced / ephemeral fan-out	Teams already running Postgres

On a standalone (non-cluster) Redis, store_event runs as a single server-side EVALSHA (counter increment plus the event write in one step) rather than an INCR followed by a pipeline, halving the per-event round-trips. The store probes for this on its first write and falls back automatically on Redis Cluster or any server without scripting, so behavior is identical either way.

Per-backend construction, configuration, write-behind tuning, and multi-tenant setup live in docs/backends.md; latency and throughput characteristics in docs/benchmarks.md.

Installation

# SQLite backend (no external service needed)
pip install "mcp-persist[sqlite]"

# Redis backend
pip install "mcp-persist[redis]"

# Postgres backend
pip install "mcp-persist[postgres]"

# Multiple backends
pip install "mcp-persist[sqlite,redis,postgres]"

# Optional extras: zstd compression, OpenTelemetry metrics export,
# AES-256-GCM encryption at rest
pip install "mcp-persist[zstd]"
pip install "mcp-persist[otel]"
pip install "mcp-persist[crypto]"

Programmatic features at a glance

Beyond drop-in resumability, every store exposes a small set of building blocks. Full API and examples in docs/api.md.

• subscribe(): push new events to an in-process consumer as they're written (Redis pub/sub, Postgres LISTEN/NOTIFY, SQLite polling).
• migrate(): copy events between backends (e.g. SQLite → Postgres as you grow), preserving per-stream ordering.
• compression="gzip" / "zstd": transparently compress large payloads above a threshold; decompression on read is automatic and config-independent. zstd (via the zstd extra) gives a better ratio for JSON-RPC.
• Encryption at rest: pass a keyring= (via the crypto extra) to AES-256-GCM encrypt payloads before they reach the backend; decryption on read is automatic and marker-driven, composes with compression, and supports zero-downtime key rotation. See docs/encryption.md.
• Multi-tenancy: bind a store to a tenant_id to isolate event streams per customer inside one shared backend (scoped reads, purge, and metrics). See docs/multi-tenancy.md.
• Per-team retention: enforce per-tenant policies (RetentionPolicy) and record deletions to an append-only audit trail (DatabaseAuditSink) via the RetentionScheduler:

  policy = RetentionPolicy(windows={"team-a": 86400, None: 3600}, default=172800)
  async with RetentionScheduler(store, policy, DatabaseAuditSink(store), interval=300):
      ...
  

• BatchingEventStore: buffer writes for high-throughput Redis/Postgres deployments, flushing on a size or latency ceiling while still returning event IDs synchronously. See docs/api.md.
• Tiered storage: archive expired events into cold storage instead of deleting them (ArchiveScheduler), and resume across both tiers (ChainedEventStore). See docs/tiered-storage.md.
• Event stream forking: branch an existing stream at any point and replay from that branch with different inputs or models (preserving the original branch intact), turning the linear log into a tree for systematic A/B evaluation. See docs/api.md.
• Metrics: pass a metrics= collector (a Protocol, the built-in LoggingMetricsCollector, or OTelMetricsCollector for OpenTelemetry) to emit to Prometheus/Datadog/etc.; zero overhead when unused. The proxy adds an optional on_proxy_replay hook for reconnect/replay rates and blocked cross-session attempts.
• PurgeScheduler: run purge_expired() on an interval for SQLite/Postgres (Redis expires natively).
• event_store_from_env(): pick the backend at deploy time from MCP_PERSIST_* env vars, no branching in code.
• ping() and health(): backend liveness/readiness probes for health endpoints; health() adds latency and backend-specific detail (DB size, memory, pool) as a HealthReport.
• export_stream() / import_stream() (and mcp-persist dump / load): capture one stream to portable JSON and restore it into a fresh store, for bug reports and test fixtures.

Architecture & guarantees

• Ordering: event IDs are monotonically increasing; replay order is preserved per-stream.
• Concurrency: duplicate IDs are structurally impossible (AUTOINCREMENT / IDENTITY / INCR); Redis and Postgres take concurrent writes, SQLite is single-writer.
• Durability: SQLite uses WAL, Postgres is ACID, Redis depends on its persistence config (use AOF for strong durability).

Full treatment, including the Redis write-ceiling caveat, in docs/architecture.md.

Examples

The examples/ directory contains minimal, runnable MCP servers: the with_persistence() one-liner, plus each backend wired manually into a real StreamableHTTPSessionManager:

File	Approach	Run
fastmcp_plugin_server.py	with_persistence() plugin (SQLite)	python examples/fastmcp_plugin_server.py
sqlite_server.py	Manual SQLiteEventStore	python examples/sqlite_server.py
redis_server.py	Manual RedisEventStore	python examples/redis_server.py
postgres_server.py	Manual PostgresEventStore	python examples/postgres_server.py

Each one is a self-contained MCP server you can connect to with any MCP client at http://localhost:8000/mcp (the three backend servers are a note-taking app; the plugin server is a minimal echo server). See examples/README.md for prerequisites, setup, and a client snippet.

Benchmarks

Measured at --events 5000 --concurrency 500 (AMD Ryzen AI 7 350, local Redis 8 / Postgres 18; indicative, not authoritative):

Backend	store throughput	replay 1,000
SQLite	23,517 ev/s	6.51 ms
Redis	7,857 ev/s	8.79 ms
Postgres	7,427 ev/s	6.58 ms

Full methodology, environment spec, percentiles, and analysis in docs/benchmarks.md. Run it yourself with uv run python benchmarks/benchmark.py --events 5000 --concurrency 500.

📚 Documentation

Guide	What's in it
docs/backends.md	Manual wiring, per-backend config, write-behind commits, multi-tenant isolation, create() lifecycle
docs/cli.md	doctor, stats, purge (incl. --older-than), dump/load & migrate full reference: sample output, --json, exit codes
docs/api.md	subscribe, migrate, export_stream/import_stream, metrics + OpenTelemetry + DEBUG_PERSIST, compression, batching, tiered storage, PurgeScheduler, env config, ping/health
docs/encryption.md	AES-256-GCM encryption at rest: KeyRing, env config, key rotation, composition with compression, threat model
docs/multi-tenancy.md	Per-tenant isolation: binding tenant_id, scoped reads/purge/metrics, how each backend isolates
docs/tiered-storage.md	Archiving expired events to cold storage: ArchiveScheduler, ChainedEventStore, resume across tiers
docs/architecture.md	Event ordering, concurrency & write semantics, consistency & durability
docs/benchmarks.md	Benchmark methodology, environment spec, full result tables
docs/production.md	Deployment topologies, sizing, failure modes, TLS/credentials, checklist
docs/typescript.md	Proxying a TypeScript (or any non-Python) MCP server

Deploying to production

Once a backend is wired in, see the production guide for operating it: scheduling purge_expired() so storage doesn't grow without bound, treating the store as a critical dependency (failure modes), pre-creating schema under restricted database permissions, TLS and credential handling, connection and pool sizing across workers, and a deployment checklist.

Development

git clone https://github.com/Ar-maan05/mcp-persist
cd mcp-persist
uv sync --all-extras --dev
uv run pytest tests/

The suite is 300+ async tests covering all three backends. The Redis tests use fakeredis and the SQLite tests use in-memory aiosqlite, so the default run needs no external servers. The Postgres tests require a real server and are skipped unless MCP_TEST_POSTGRES_URL is set; to run them and the Redis suite against real backends:

MCP_TEST_REDIS_URL=redis://localhost:6379/0 \
MCP_TEST_POSTGRES_URL=postgresql://postgres@localhost:5432/postgres \
uv run pytest tests/

See CONTRIBUTING.md for more.

License

MIT