axonflow-litellm
AxonFlow governance integration for LiteLLM. Enforce policies, audit LLM calls, and gate high-risk requests behind human approval — all through a drop-in wrapper around litellm.completion().
Installation
pip install axonflow-litellm
Quick Start
from axonflow_litellm import AxonFlowLogger, AxonFlowLoggerConfig, PolicyDeniedError
logger = AxonFlowLogger(
AxonFlowLoggerConfig(
endpoint="http://localhost:8080",
client_id="my-app",
client_secret="...",
)
)
try:
response = logger.completion(
model="gpt-4o",
messages=[{"role": "user", "content": "Summarize quarterly earnings"}],
)
print(response.choices[0].message.content)
except PolicyDeniedError as e:
print(f"Blocked: {e.reason}")
Hosted Starter account and credential
For Hosted Starter, start in the verified Portal. After the owner activates a plan, download the Hosted API link command, run it locally, and approve its device code in the signed-in owner session. The protected ~/.config/axonflow/hosted-api.json file supplies the endpoint, clientId for client_id, and clientSecret for client_secret. Keep that file and secret out of prompts and source control. This integration consumes a linked client credential; it does not create a tenant through anonymous /api/v1/register.
For governed Hosted calls, use AxonFlowLoggerConfig(fail_open=False) with logger.completion() or logger.acompletion(). An answered 402 hosted_payment_required or 429 hosted_monthly_limit_reached is a refusal regardless of the fail mode; restore the plan or wait for the allowance reset before retrying. Ordinary capacity 429s use the configured availability fallback. LiteLLM's audit-only callback cannot block a call, so it is not an enforcement path.
How It Works
AxonFlowLogger provides two integration modes:
Governance Mode (recommended)
Use logger.completion() or logger.acompletion() as drop-in replacements for litellm.completion() / litellm.acompletion():
- Pre-check — sends the prompt to AxonFlow for policy evaluation
- HITL — if the policy returns
require_approval, creates a human-in-the-loop review request and polls until approved, rejected, or timed out - LLM call — delegates to LiteLLM (all providers supported)
- Audit — records the response to AxonFlow for observability
# Async (recommended for production)
response = await logger.acompletion(
model="claude-sonnet-4-6",
messages=[{"role": "user", "content": "..."}],
user_token="jwt-from-your-auth",
)
Audit-Only Mode
Register as a LiteLLM callback for observability without blocking:
import litellm
litellm.callbacks = [logger]
response = litellm.acompletion(model="gpt-4o", messages=[...])
In this mode, every LLM call is recorded to AxonFlow for audit trail. Policy denials are logged as warnings but cannot block the request (a LiteLLM SDK limitation — callback exceptions are silently swallowed).
Configuration
| Parameter | Default | Description |
|---|---|---|
endpoint |
(required) | AxonFlow agent URL |
client_id |
(required) | AxonFlow client identifier |
client_secret |
"" |
AxonFlow client secret |
default_user_token |
"anonymous" |
Token for policy evaluation when none provided — accepted only by community-mode deployments; see User tokens |
tenant_id |
None |
AxonFlow tenant identifier |
fail_open |
True |
Allow LLM calls when AxonFlow is unreachable |
call_timeout_seconds |
5.0 |
Per-hook timeout for AxonFlow API calls |
breaker_failure_threshold |
5 |
Consecutive failures before circuit opens |
breaker_recovery_seconds |
30.0 |
Wait before attempting recovery probe |
enable_hitl_polling |
True |
Enable HITL approval flow for require_approval |
approval_poll_interval_seconds |
2.0 |
Polling interval for HITL status |
approval_max_wait_seconds |
300.0 |
Maximum wait for HITL decision |
extra_context |
{} |
Additional context sent with every pre-check |
User tokens: community vs. enterprise
What user_token must be depends on the AxonFlow deployment mode:
-
Community / community-SaaS — the per-user token is not validated. The
default_user_token="anonymous"placeholder can be used with an otherwise valid client configuration; a new paid Hosted account still needs its separately linked client ID and secret. -
Enterprise / evaluation — the platform validates
user_tokenon/api/policy/pre-checkand rejects both absent and invalid tokens (including the"anonymous"placeholder) with 401. Pass a real per-user token on every call (or setdefault_user_tokento one):response = await logger.acompletion( model="gpt-4o", messages=[...], user_token=minted_token, # per-user token minted by your admin )
Admins mint per-user tokens via the customer-portal admin API (
POST /api/v1/admin/organizations/{org_id}/user-tokens) — see the per-user token provisioning guide. Admin-minted (HS256) tokens only: the pre-check plane pins the accepted algorithm to HS256, so tenant-OIDC access tokens (RS256) are rejected there. The audit trail then attributes each LLM call to that user.
A platform rejection (401/402/403 — bad credentials, rejected
user_token, budget block, tenant mismatch) raises PolicyDeniedError
whenever the pre-check reaches the platform, regardless of fail_open.
fail_open covers availability only: a healthy platform refusing the
request is a governance verdict, not an outage, and proceeding would
silently skip governance on every call. (In audit-only callback mode the
error is logged but cannot block — a LiteLLM callback limitation.)
Fail-Open vs. Fail-Closed
By default, fail_open=True: if AxonFlow is unreachable or times out, the LLM call proceeds normally. This ensures an AxonFlow outage does not break your application.
For high-stakes workloads where unapproved LLM calls must never proceed:
config = AxonFlowLoggerConfig(
endpoint="http://localhost:8080",
client_id="payments-service",
client_secret="...",
fail_open=False,
)
Sync vs. Async
Both litellm.completion() (sync) and litellm.acompletion() (async) are fully supported.
When registered via litellm.callbacks, sync hooks delegate to their async counterparts via asyncio.run(). This adds minor overhead (~1ms) per hook call in the sync path. For performance-critical sync workloads, use logger.completion() directly (governance wrapper) which amortizes the event loop creation.
If sync hooks are invoked inside a running event loop (unusual — e.g., sync callbacks from an async framework), a one-time RuntimeWarning is emitted directing you to acompletion().
Sync callback mode caveats
In sync callback mode (litellm.callbacks = [logger] + litellm.completion()), each callback hook creates an ephemeral asyncio event loop via asyncio.run(). Pre-check (governance) and post-LLM audit both fire and write to AxonFlow. However:
- Audit write failures are logged at WARNING level and do not raise to the caller (fail-open by default). If AxonFlow is temporarily unreachable during the audit phase, the LLM response is still returned but the audit row may be missing.
- Each hook creates a new event loop, so connection pooling is not shared across hooks within the same LLM call. This is slightly less efficient than the governance wrapper path.
For strict audit guarantees (every LLM call audited, failure = exception), use logger.completion() or logger.acompletion() instead of the callback registration path.
Exceptions
| Exception | When |
|---|---|
PolicyDeniedError |
Policy denied the request |
ApprovalRejected |
HITL approval was rejected |
ApprovalTimeout |
HITL approval timed out |
All exceptions carry .reason (string) and .policies (list of policy IDs).
These exceptions do NOT extend litellm.exceptions.APIError — catch governance denials via PolicyDeniedError, not LiteLLM's exception hierarchy.
MCP Governance
LiteLLM is LLM-completion-focused. For MCP tool governance, use AxonFlow's MCP server directly.
Requirements
- Python >= 3.10
litellm>= 1.40axonflow>= 9.3.0
Telemetry
This integration declares itself on the AxonFlow SDK's existing anonymous heartbeat, so aggregate adoption figures can tell LiteLLM-governed usage apart from bare SDK usage. Without it the two are indistinguishable: both report the same sdk, the same sdk_version and the same endpoint.
It adds no network request. adapter:litellm rides the features array of the heartbeat the SDK already sends — there is no second ping, no second endpoint, and no new configuration surface. The declaration happens once, immediately before the AxonFlow client is constructed, so it reaches the very first heartbeat.
What is and is not collected: the string litellm, and nothing else. No prompts, no completions, no model names, no user identities, no configuration. Everything else on the heartbeat is the SDK's own — see the AxonFlow Python SDK's telemetry section for the full field list and the opt-out.
AXONFLOW_TELEMETRY=off suppresses the heartbeat, and this declaration with it. An SDK older than 9.3.0 simply has nothing to declare to; the integration works normally either way.
License
MIT
Metadata
Release files for axonflow-litellm 1.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| axonflow_litellm-1.1.1.tar.gz | 34.4 kB | Without provenance | Details for axonflow_litellm-1.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| axonflow_litellm-1.1.1-py3-none-any.whl | Python 3 | none | any | Details for axonflow_litellm-1.1.1-py3-none-any.whl |
Total release size: 54.3 kB
Release files / axonflow_litellm-1.1.1.tar.gz
| Download URL | axonflow_litellm-1.1.1.tar.gz |
|---|---|
| Size | 34.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1a52831d208f48c9760c850b0dfabf8f29a503b0801a32f558757ce8eb930011
|
|
BLAKE2b-256 checksum How to use checksums |
3e5b27e79b10951bffb74a8b266879687ccc402ea2ec1d579e4ec0a7a5aa7546
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.14
|
Release files / axonflow_litellm-1.1.1-py3-none-any.whl
| Download URL | axonflow_litellm-1.1.1-py3-none-any.whl |
|---|---|
| Size | 20.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
688ed9ab7e39bcf2f2a5d00dcd79fa0cf699dcbeb9f0e72bce7122398318c637
|
|
BLAKE2b-256 checksum How to use checksums |
2e37c2c9ecfede61411f45033fb982e3720ae01b8d3a3160e6b64018aa98ecc2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.14
|
Security for axonflow-litellm 1.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.