Agent Harness

A terminal-based ReAct agent for the University of Passau AI Engineering Lab (SoSe 2026). The Week 3 harness adds specialised sub-agents, config-driven permission tiers, and Prometheus/Grafana observability to the Week 2 base of workspace-confined filesystem tools, runtime-discovered MCP tools, and persistent long-term memory — all authority stays in deterministic local code.

The LLM proposes. The local registry validates, authorizes, executes, observes, and audits. A sub-agent is the same loop, narrower.

The features are built around one product: an operations copilot for a farm's wind turbine and solar arrays — see the problem statement. The sub-agents have real roles because there is real analysis to delegate; the require-confirmation tier gates real consequences (restart an inverter, schedule turbine curtailment); the dashboard shows real runs.

First time here? Follow Getting started. For complete engineering context, read Project engineering reference.

Requirements

• Python 3.12
• An InnKube inference access key
• Docker (recommended, but optional for local development); Docker Compose for the optional Prometheus/Grafana stack
• Network access to the configured LLM; to Open-Meteo for the plant and hourly forecast tools; and to the US National Weather Service for the NWS tools. PLANT_OFFLINE=1 runs the plant on synthetic weather with no network.

Architecture

The non-root Python process runs a bounded reason → act → observe loop. Native filesystem and memory tools, the delegate tool, and MCP-discovered tools all enter the same ToolRegistry, argument-validation, permission, observation, and audit path. A sub-agent is that same loop with its own instructions, its own limits, and an allow-listed view of the registry whose tiers can only be tighter than the parent's. Every audit event fans out to a JSONL file and a Prometheus exporter; a compose-managed Prometheus scrapes it over a shared Docker network and Grafana shows it. The bundled MCP servers are stdio subprocesses: the extended weather quickstart, the unmodified third-party mcp-server-time, and the plant operations server, which reads a SunSpec inverter fleet over Modbus TCP.

The maintainable diagram source is docs/architecture/week-3.svg. Earlier diagrams are retained in docs/architecture/ so each week's architectural change stays visible.

Quick start

cp .env.example .env                  # put your InnKube key in AGENT_API_KEY
docker network create agentnet        # once; ./agent also creates it
docker compose up -d                  # Prometheus :9090, Grafana :3000 (optional)
./agent --verbose                     # builds the image, joins agentnet, enters agent mode

Then open http://localhost:3000/d/agent-harness and follow the Week 3 demo.

Week 3 capabilities

• Specialised sub-agents on the same harness. Roles are configuration: a name, instructions, a tool allow-list, per-tool tiers that can only tighten the parent's, and their own iteration and observation limits. The parent delegates with one delegate tool; the child's answer is the parent's observation; every child event is stamped with the role, a delegation id, and the parent's turn. Delegation is one level deep by construction.
• Permission management for built-in and MCP tools. allow, deny, and ask (require confirmation) through one path. Built-in tiers are re-tierable in config; MCP tiers stay with their server; a base deny — workspace confinement, unknown action, server deny — is never lifted by an override.
• Observability. Metrics are a projection of the audit stream: turns, active runs, latencies, success/failure, LLM tokens, tool usage, permission decisions, memory operations, sub-agent activity, each with a scope of parent or the role. /metrics on agent.metrics_port; Prometheus and a provisioned 13-panel Grafana dashboard via compose.yaml; the JSONL audit is the per-run trace.
• The plant. A SunSpec/Modbus TCP inverter fleet (simulated in-process, reached over TCP by a real Modbus client), weather-normalised diagnosis with fault attribution priced in euros, the day-to-day comparison that separates weather from faults, a bat-curtailment planner that applies the permit's thresholds to tonight's forecast and shows every one, and two approval-gated control actions.
• Week 2 hardening. Tool observations are capped before entering the context; the memory store is tenant-scoped with session provenance; MCP adaptation failure is isolated per server; structured MCP results reach the model with their values; the weather server gained a global hourly forecast from Open-Meteo.
• Week 1 and Week 2 behaviour — filesystem confinement, one-use overwrite approval, runtime MCP discovery, long-term memory across restarts — is unchanged and still tested.

Configuration

The runnable default is config/config.example.yaml. The Week 3 settings:

agent:
  metrics_port: 9464                 # Prometheus scrape endpoint; omit to disable
  instructions: |                    # appended to the built-in guidance for this deployment
    You are the operations copilot for a farm's wind turbine and solar arrays. ...
  # tool_permissions:                # re-tier any tool by public name; a base deny is final
  #   modify_file: allow

mcp:
  servers:
    - name: "plant"
      command: "python3"
      args: ["-m", "mcp_servers.plant.server"]
      env: { PLANT_SITE_FILE: "mcp_servers/plant/site.example.json" }
      default_permission: "allow"
      tool_overrides:
        restart_inverter: "ask"
        set_turbine_curtailment: "ask"
        clear_turbine_curtailment: "ask"

sub_agents:
  - name: "diagnostician"
    description: "Reads plant telemetry ... attributes underperformance to a cause with its cost."
    instructions: |
      You are the plant diagnostician ...
    tools: ["mcp__plant__list_assets", "mcp__plant__diagnose_performance",
            "mcp__plant__read_inverter", "mcp__plant__compare_days", "recall"]
    max_iterations: 6
  - name: "compliance"
    ...

AGENT_METRICS_PORT, AGENT_MEMORY_DB, and AGENT_AUDIT_LOG override their YAML counterparts. The plant server reads PLANT_SITE_FILE, PLANT_MODBUS_HOST/PORT, PLANT_SIMULATOR (0 to point at real hardware), and PLANT_OFFLINE.

Capability and permission boundary

Native tools: create_folder, create_file, navigate, search_in_files, read_file, modify_file; with memory configured, remember and recall; with roles configured, delegate. Discovered MCP tools are added as mcp__<server>__<tool> — never as bare names, never through a side channel.

Permission precedence, most specific last:

workspace confinement / unknown action ─▶ server default ─▶ server tool_overrides
        ─▶ agent.tool_permissions ─▶ sub-agent role tool_permissions (tighten only)

• Workspace reads, searches, navigation, and non-overwriting creation are allowed inside the canonical workspace; modify_file asks, because it overwrites bytes.
• remember is covered by the one-time memory consent; recall and delegate are allowed — the sub-agent's own tools are gated separately.
• Plant reads and diagnosis are allowed; restart_inverter, set_turbine_curtailment, and clear_turbine_curtailment ask.
• A sub-agent's ask reaches the same human prompt, labelled with the role. A role can make a tool stricter than the parent's tier, never looser.
• Unknown tools, unknown roles, and unavailable approvals fail closed.

There is no delete, rename, shell, arbitrary-code, or unrestricted-HTTP tool. This is a controlled harness, not a hostile-code sandbox. See Security before mounting sensitive data or configuring a new MCP command.

Tests and quality checks

ruff check .
ruff format --check .
mypy src tests mcp_servers
pytest                       # 174 deterministic cases, incl. real Modbus TCP round trips

Real-LLM end-to-end tests are opt-in because they use credentials, network, and API budget:

RUN_E2E=1 AGENT_CONFIG=config/config.yaml pytest -m e2e tests/e2e

The Week 3 E2E scenario asks a real model to find the money-losing inverter and restart it, and asserts that the diagnosis was delegated, that the restart reached the approval callback before the device, and that both the audit trail and the Prometheus registry recorded it.

Documentation

• Documentation index: every document, grouped by week and topic
• Problem statement: the user, the problem, the decisions
• Week 3 implementation: §8.4 requirements → code → evidence, with every design decision and its reason
• Week 3 live demo: the delegated diagnosis, the inversion, the approved restart, the permit check, memory, and the dashboard
• Week 3 demo scenarios: cards mapped to automated tests
• Week 2 implementation and scenarios: retained and still passing
• Implementation status: implemented, deferred, limitations
• Security: controls, trust boundaries, audit events, risks
• Lab specification: the graded requirements, converted from the course PDF
• Getting started and the Project engineering reference

Project layout

agent, compose.yaml              One-command launcher; Prometheus + Grafana stack
config/                          Runnable validated configuration
docker/                          Non-root Python 3.12 runtime image
observability/                   Prometheus scrape config, Grafana provisioning, dashboard
docs/                            Index, security, status, problem statement
docs/week-1/ … docs/week-3/      Per-week demo, presentation, and evidence docs
docs/architecture/               Diagrams and re-export instructions
docs/lab_instructions/           The graded course specification
mcp_servers/weather/             Extended official weather quickstart + Open-Meteo forecast
mcp_servers/plant/               SunSpec/Modbus simulator and gateway, diagnosis, curtailment
mcp_servers/common/              Open-Meteo client shared by the servers
src/enzomotive/               Loop, LLM client, config, audit, UI, memory
src/enzomotive/subagents.py   Delegation runner: one role, one narrower loop
src/enzomotive/observability.py  Prometheus sink over the audit stream
src/enzomotive/mcp/           MCP configuration, lifecycle, adapter, errors
src/enzomotive/tools/         Registry and sub-agent views, permission tiers, filesystem
tests/unit/                      Fast deterministic harness tests
mcp_servers/*/tests/             Server-side tests, incl. Modbus TCP round trips
tests/e2e/                       Opt-in real-LLM workflows