fred-runtime
fred-runtime is the infrastructure and execution layer for Fred agent pods.
It wires platform services (databases, LLM routing, MCP, observability, security) into
the execution engine defined by fred-sdk, and provides a ready-to-use FastAPI app
factory so any agent pod is just a registry and a YAML file away from running.
Where fred-runtime fits
fred-pod configuration, identity, naming
└── fred-sdk authoring contracts
└── [agents] + fred-core, langchain, langgraph
└── fred-runtime execution, MCP, model routing ← this package
└── [app] + the FastAPI pod factory → an agent pod
Rule of thumb:
- Write agent logic against
fred-sdk. - Write infrastructure adapters (DB, MCP server, Keycloak, object store) in
fred-runtime. fred-sdkmust stay importable on a bare laptop with no services running.
A Knowledge Base pod does not need this package. What a pod is, and which package each kind installs: fred-pod.
What's in the box
fred_runtime.app — Agent pod factory
The main entry point for building a Fred agent pod.
from fred_runtime.app import create_agent_app, load_agent_pod_config
config = load_agent_pod_config() # reads ENV_FILE + CONFIG_FILE (configuration.yaml)
app = create_agent_app(registry=REGISTRY, config=config)
create_agent_app returns a FastAPI application that exposes:
| Method | Path | Description |
|---|---|---|
POST |
{base_url}/agents/execute |
Single-turn execution — returns final JSON |
POST |
{base_url}/agents/execute/stream |
Streaming SSE execution — yields RuntimeEvent objects |
GET |
{base_url}/agents |
List registered agent IDs |
GET |
{base_url}/agents/sessions |
List session IDs for a user |
GET |
{base_url}/agents/sessions/{id}/messages |
Full conversation history for a session |
GET |
/v1/models |
OpenAI model list (agent IDs as model names) |
POST |
/v1/chat/completions |
OpenAI chat completions — works with Open WebUI, openai-python SDK, etc. |
The OpenAI-compatible /v1 surface is enabled by default.
Set app.openai_compat: false in configuration.yaml to disable it for internal pods.
Multi-turn continuity and HITL (human-in-the-loop) are handled transparently through
the SQL checkpointer. The session ID is the LangGraph thread_id.
fred_runtime.runtime_support — Infrastructure adapters
| Module | What it provides |
|---|---|
sql_checkpointer |
Durable LangGraph checkpointer backed by SQLite (dev) or PostgreSQL (prod) |
user_token_refresher |
Transparent Keycloak token refresh for long-lived agent sessions |
request_context_helpers |
FastAPI dependency helpers for extracting user/session context |
fred_runtime.model_routing — LLM routing
Multi-provider model resolution with per-agent tuning overrides.
Supports routing by agent ID, execution category, or explicit model name.
Providers: OpenAI, Azure OpenAI, Mistral, Ollama, and any LangChain-compatible backend.
fred_runtime.common — Knowledge Flow + MCP clients
HTTP clients that connect agent tools to the Fred platform services:
| Client | Connects to |
|---|---|
kf_http_client |
Knowledge Flow REST API (generic) |
kf_vectorsearch_client |
Vector search / retrieval |
kf_markdown_media_client |
Document content (Markdown + media) |
kf_workspace_client |
Workspace and library management |
kf_fast_text_client |
FastText classification |
mcp_runtime / mcp_toolkit |
MCP server lifecycle and tool injection |
context_aware_tool |
Tool base class that propagates the runtime context (user, team, token) |
fred_runtime.integrations — v2 runtime adapters
Small adapters that bridge the platform-agnostic fred-sdk contracts to real
services available at runtime (chat model factory, checkpointer wiring, MCP discovery).
fred_runtime.client — Developer CLI (fred-agents-cli)
An interactive REPL and one-shot client for any Fred agent pod:
# Interactive mode — connects to http://127.0.0.1:8000/api/v1 by default
fred-agents-cli
# Set the current team scope inside the REPL
/team my-team
# Inspect runtime KPIs without Grafana
/kpi
/kpi tool_name=search
# One-shot
fred-agents-cli --agent my-agent "What is the status of cluster A?"
# Keycloak browser login
fred-agents-cli --login
# Start already scoped to one team
fred-agents-cli --team-id my-team
# Override the metrics endpoint used by /kpi
fred-agents-cli --metrics-url http://127.0.0.1:9115/metrics
The target pod URL is resolved from configuration.yaml automatically,
or overridden with --base-url / FRED_AGENT_POD_URL.
Configuration
Every Fred pod uses the same two-file convention:
| File | Purpose |
|---|---|
.env (path from ENV_FILE) |
Secrets: API keys, DB URLs, Keycloak credentials |
configuration.yaml (path from CONFIG_FILE) |
App settings: port, base URL, LLM routing, observability, security |
Minimal configuration.yaml for a local pod:
app:
name: "My Agent Pod"
base_url: "/myapp/v1"
host: "0.0.0.0"
port: 8010
log_level: "info"
limit_concurrency: 200
metrics_address: "127.0.0.1"
metrics_port: 9115
kpi_process_metrics_interval_sec: 10
observability:
tracer: logging
metrics: prometheus
ai:
knowledge_flow_url: "http://localhost:8111/knowledge-flow/v1"
Full schema: fred_runtime.app.config.AgentPodConfig.
When observability.metrics: prometheus is enabled, create_agent_app(...)
starts a dedicated Prometheus exporter on app.metrics_address:app.metrics_port
and restores the shared Fred KPI pipeline, including process and SQL pool KPIs.
Set app.limit_concurrency: null to disable Uvicorn connection limiting, or a
positive integer to reject excess concurrent HTTP and WebSocket connections
with 503 before application code runs.
Installation
# Core library (runtime adapters, CLI, model routing)
pip install fred-runtime
# With FastAPI pod factory
pip install fred-runtime[app]
fred-runtime depends on fred-sdk[agents], so an agent pod installs nothing
else to get the authoring surface. A Knowledge Base pod installs
fred-sdk[knowledge-base] instead and never needs fred-runtime.
Requires Python 3.12.
Building an agent pod
A minimal pod is three files:
main.py
from fred_runtime.app import create_agent_app, load_agent_pod_config
from myapp.registry import REGISTRY
config = load_agent_pod_config()
app = create_agent_app(registry=REGISTRY, config=config)
__main__.py
import uvicorn
from fred_runtime.app import load_agent_pod_config
def main():
config = load_agent_pod_config()
uvicorn.run(
"myapp.main:app",
host=config.app.host,
port=config.app.port,
limit_concurrency=config.app.limit_concurrency,
reload=True,
)
if __name__ == "__main__":
main()
registry.py
from fred_sdk import ReActAgent
class MyAgent(ReActAgent):
agent_id: str = "acme.my.assistant"
role: str = "Assistant"
description: str = "Answers general questions."
system_prompt_template: str = "You are a concise assistant."
MY_AGENT = MyAgent()
REGISTRY = {MY_AGENT.agent_id: MY_AGENT}
How to write agents and their tools: the fred-sdk README. fred-samples is a working pod with several agents.
Related packages
| Package | PyPI | Role |
|---|---|---|
fred-pod |
pypi | What every pod shares — configuration, identity, naming |
fred-core |
pypi | The agents platform — stores, model providers, logging, observability |
fred-sdk |
pypi | Authoring — agents, tools, Knowledge Bases |
fred-runtime |
pypi | This package |
License
2 maintainers
Metadata
2 maintainers
Release files for fred-runtime 4.5.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| fred_runtime-4.5.1.tar.gz | 888.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| fred_runtime-4.5.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.4 MB
Release files / fred_runtime-4.5.1.tar.gz
| Download URL | fred_runtime-4.5.1.tar.gz |
|---|---|
| Size | 888.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
34264c6f14285e247d95b75bc11fc6bc9121364b37feae0ab3fd7e8143ed3548
|
|
BLAKE2b-256 checksum How to use checksums |
1215b40f8c85b06b82214c7edb96e11d0cae696cacd44ffcbbc1776d8054ac40
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / fred_runtime-4.5.1-py3-none-any.whl
| Download URL | fred_runtime-4.5.1-py3-none-any.whl |
|---|---|
| Size | 550.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ead433fe277c21303bef25c4e0ce9af8ed08f7e4347b04af727edb1a9351848e
|
|
BLAKE2b-256 checksum How to use checksums |
c9e98c3c2306d236570d40a1416ce50561e4f1f5d774b9c22147eb2c3bf53834
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
