litellm-dynamo

Meter and enforce budgets on every model call through your LiteLLM proxy — without adding a network hop to the request path. You operate the gateway; Dynamo Protocol gives each caller a capped, revocable allowance; this plugin makes your proxy honor it: no live credential, no completion (HTTP 402).

Payments are none of the plugin's business: the credential might be backed by a no-money control ledger, stablecoin escrow, or a card authorization-backed budget — the plugin neither knows nor cares (mode-agnostic by construction).

Install

pip install "litellm-dynamo[litellm]"    # Python ≥ 3.10; LiteLLM pin 1.61.20

Two-minute quickstart

1. Point it at your engine host — two env vars are mandatory:

export DYNAMO_ENGINE_URL=http://127.0.0.1:8500     # a running dynamo-core
export DYNAMO_ENGINE_SIGNER=0xYourEngineSigner     # PINNED signer; verification is offline

2. Choose your token→units mapping (config, not code):

export DYNAMO_UNITS_PER_PROMPT_TOKEN=1
export DYNAMO_UNITS_PER_COMPLETION_TOKEN=2
export DYNAMO_UNITS_PER_REQUEST=0

3. Register the hook. dynamo_hook.py next to your proxy config:

from litellm_dynamo import DynamoLiteLLMHook, config_from_env
dynamo_hook = DynamoLiteLLMHook(config_from_env())

and in the LiteLLM config:

litellm_settings:
  callbacks: dynamo_hook.dynamo_hook

4. Run the background loop (sync + async metering — the request path never blocks on the network):

import threading, time
def loop(hook, stream_ids):
    while True:
        for sid in stream_ids():
            try: hook.guard.sync_stream(sid)
            except Exception: pass   # cache ages toward DENY — fail closed
        hook.guard.flush_metering()
        time.sleep(10)               # keep < DYNAMO_MAX_STALE_SECONDS / 2
threading.Thread(target=loop, args=(dynamo_hook, my_stream_ids), daemon=True).start()

Callers attach their allowance credential per request:

curl http://localhost:4000/v1/chat/completions \
  -H "x-dynamo-allowance: <compact-JWS credential>" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}]}'

What it enforces (all fail closed)

Verification is offline (JWS against the pinned signer) and metering is async — the request path gains zero network round-trips.

What this does NOT do

• It moves no money. Enforcement and settlement live in the caller's Dynamo deployment; Control Mode credentials are free enforcement — caps and a signed meter, no payments.
• It never reads, stores, or transmits prompt/response content to the metering side: metering carries {units, requestRef} and nothing else.

Docs: https://github.com/DynamoProtocol/dynamo-open · License: Apache-2.0