moesif-litellm

Moesif plugin for LiteLLM. Works in both SDK mode and proxy mode.

Moesif is an API analytics and governance platform. This plugin automatically captures LLM requests and responses so you can monitor usage, track costs, enforce governance rules, and analyze traffic by user or company with no changes to your LLM calls.

What it does

• Logging: captures every LLM request and response and sends to Moesif for analytics, cost tracking, and user insights
• Governance: enforces block rules configured in the Moesif dashboard; requests are stopped before they reach the LLM
• Sampling: controls what percentage of traffic is logged; sampled events are weighted so Moesif can extrapolate totals correctly
• Identity resolution: automatically resolves user_id and company_id from multiple sources (kwargs, virtual keys, JWT, callbacks)
• Body masking: redacts sensitive fields in request/response bodies before logging
• Event filtering: skip_event callback to drop specific events from being logged
• Event mutation: mask_event_model callback to transform the event before it is sent

How it works

Events are queued in memory and flushed to POST /v1/events/batch in the background. Failed batches are re-queued automatically. Governance rules are fetched from Moesif /v1/rules with ETag caching. In sync mode (litellm.completion) each event is sent immediately via a blocking HTTP call. In async mode (litellm.acompletion / proxy) events are batched and flushed on a background timer. See batch_size and flush_interval in the Configuration reference.

---

Prerequisites

• Python >= 3.10
• LiteLLM >= 1.82.0

Installation

pip install moesif-litellm

---

SDK Mode

Attach the handler directly to LiteLLM callbacks in your Python code.

import litellm
from moesif_litellm import MoesifHandler

litellm.callbacks = [MoesifHandler()]

litellm.completion(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    user="alice",                          # → user_id in Moesif
    metadata={"company_id": "acme-corp"},  # → company_id in Moesif
)

Set your Moesif application ID:

export MOESIF_APPLICATION_ID=your-moesif-app-id

Or pass it directly:

MoesifHandler(application_id="your-moesif-app-id")

For custom identity resolution:

MoesifHandler(
    identify_user=lambda kwargs, payload: current_user.id,
    identify_company=lambda kwargs, payload: current_user.company_id,
)

Note: In SDK mode there is no HTTP request, so Moesif logs the request URI as litellmsdk/{call_type} (e.g. litellmsdk/completion, litellmsdk/embedding). In proxy mode the actual request path (e.g. /v1/chat/completions) is used.

See full example: examples/sdk/basic_usage.py

---

Proxy Mode

Run LiteLLM as a proxy server and load the plugin via a callback shim.

1. Create moesif_callback.py in the same directory as your proxy config:

from moesif_litellm import MoesifHandler
moesif_handler = MoesifHandler()

2. Reference it in proxy_config.yaml:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  callbacks: ["moesif_callback.moesif_handler"]

3. Start the proxy:

export MOESIF_APPLICATION_ID=your-moesif-app-id
litellm --config proxy_config.yaml --port 4000

4. Make requests:

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}], "user": "alice", "metadata": {"company_id": "acme-corp"}}'

Virtual keys and teams (recommended for proxy)

Assign LiteLLM virtual keys to teams, the team ID maps to company_id in Moesif automatically, no metadata needed in every request. Requests using sk-xxxx automatically get company_id="acme-corp" in Moesif. See full examples: examples/proxy/

---

Identity resolution

First non-None value wins.

user_id

Priority	Source
1	identify_user callback
2	user_api_key_end_user_id (proxy virtual key)
3	user= kwarg
4	end_user in logging payload
5	user_api_key_user_id (proxy key owner)
6	metadata.user_id
7	JWT claim (authorization_user_id_field)

company_id

Priority	Source
1	identify_company callback
2	user_api_key_team_id (proxy virtual key team)
3	requester_metadata.company_id
4	metadata.company_id
5	JWT claim (authorization_company_id_field)

---

Configuration reference

Parameter	Default	Description
application_id	$MOESIF_APPLICATION_ID	Moesif Application ID (required)
batch_size	100	Events per flush
flush_interval	2	Seconds between flushes
max_queue_size	50000	Max in-memory events
capture_request_body	True	Log request body
capture_response_body	True	Log response body
request_max_body_size	100000	Max request body bytes
response_max_body_size	100000	Max response body bytes
request_body_masks	[]	Request body keys to null out
response_body_masks	[]	Response body keys to null out
request_header_masks	[]	Request headers to remove
response_header_masks	[]	Response headers to remove
identify_user	None	(kwargs, payload) -> str
identify_company	None	(kwargs, payload) -> str
authorization_user_id_field	"sub"	JWT claim for user ID
authorization_company_id_field	None	JWT claim for company ID
skip_event	None	(kwargs, event) -> bool — return True to drop
mask_event_model	None	(kwargs, event) -> event — mutate before send

---

Masking sensitive data

Redact fields from request/response bodies or headers before they are sent to Moesif.

MoesifHandler(
    request_body_masks=["messages"],
    response_body_masks=["choices"],
    request_header_masks=["authorization"],
)

---

Filtering events

Drop specific events from being logged to Moesif entirely.

# Only log errors
MoesifHandler(skip_event=lambda kwargs, event: event["response"]["status"] == 200)

---

Sampling

Sample rates are primarily managed from the Moesif dashboard, you can set per-user and per-company rates under Governance Rules without any code changes. The sample_rate option here is a local fallback for when no dashboard rule applies:

MoesifHandler(sample_rate=10)  # fallback: capture 10% of traffic if no Moesif rule applies

If not set, defaults to 100 (all traffic logged).

---

License

Apache 2.0 - see LICENSE