agentops-cockpit 0.2.2

Production-grade Agent Operations (AgentOps) Platform

🕹️ AgentOps Cockpit

"Infrastructure gives you the pipes. We give you the Intelligence."

The developer distribution for building, optimizing, and securing AI agents on Google Cloud.

---

📽️ The Mission

Most AI agent templates stop at a single Python file and an API key. The AgentOps Cockpit is for developers moving into production. While optimized for ADK, it provides framework-agnostic governance, safety, and cost guardrails for the entire agentic ecosystem—from CrewAI to LangGraph. Based on the Google Well-Architected Framework for Agents.

---

🏗️ The Agentic Trinity

We divide the complexity of production agents into three focused pillars:

• ⚙️ The Engine: The reasoning core. Built with ADK, FastAPI, and Vertex AI.
• 🎭 The Face: The user experience. Adaptive UI surfaces and GenUI standards via the A2UI spec.
• 🕹️ The Cockpit: The operational brain. Cost control, semantic caching, shadow routing, and adversarial audits.

---

🌐 Framework Agnostic Governance

The Cockpit isn't just for ADK. It provides Best Practices as Code across all major agentic frameworks:

Whether you are building a swarm in CrewAI, a Go-based high-perf engine, or a Streamlit dashboard, the Cockpit ensures your agent maps to the Google Well-Architected Framework.

---

🚀 Key Innovation: The "Intelligence" Layer

🛡️ Red Team Auditor (Self-Hacking)

Don't wait for your users to find prompt injections. Use the built-in Adversarial Evaluator to launch self-attacks against your agent, testing for PII leaks, instruction overrides, and safety filter bypasses.

🧠 Hive Mind (Semantic Caching)

Reduce LLM costs by up to 40%. The Hive Mind checks for semantically similar queries in 10ms, serving cached answers for common questions without calling the LLM.

🏛️ Arch Review & Framework Detection

Every agent in the cockpit is graded against a framework-aware checklist. The Cockpit intelligently detects your stack—Google ADK, OpenAI Agentkit, Anthropic Claude, Microsoft AutoGen/Semantic Kernel, AWS Bedrock Agents, or CopilotKit—and runs a tailored audit against corresponding production standards. Use make arch-review to verify your Governance-as-Code.

🧗 Quality Hill Climbing (ADK Evaluation)

Following Google ADK Evaluation best practices, the Cockpit provides an iterative optimization loop. make quality-baseline runs your agent against a "Golden Dataset" using LLM-as-a-Judge scoring (Response Match & Tool Trajectory), climbing the quality curve until production-grade fidelity is reached.

---

⌨️ Quick Start

You don't even need to clone the repo to start auditing.

# 1. Audit your existing agent design
uvx agent-ops-cockpit arch-review

# 2. Stress test your endpoint
uvx agent-ops-cockpit load-test --requests 100 --concurrency 10

# 3. Scaffold a new Well-Architected app
uvx agent-ops-cockpit create my-agent --ui a2ui

---

📊 Local Development

The Cockpit provides a unified "Mission Control" to evaluate your agents instantly.

make audit-all         # 🕹️ Run ALL audits and generate a Final Report
make reliability       # 🛡️ Run unit tests and regression suite
make dev               # Start the local Engine + Face stack
make arch-review   # 🏛️ Run the Google Well-Architected design review
make quality-baseline # 🧗 Run iterative 'Hill Climbing' quality audit
make audit         # 🔍 Run the Interactive Agent Optimizer
make red-team      # Execute a white-hat security audit
make deploy-prod   # 🚀 1-click deploy to Google Cloud

---

🧭 Roadmap

• One-Click GitHub Action: Automated audits on every PR.
• Multi-Agent Orchestrator: Support for Swarm/Coordinator patterns.
• Visual Mission Control: Real-time observability dashboard.

View full roadmap →

---

🤝 Community

• Star this repo to help us build the future of AgentOps.
• Join the Discussion for patterns on Google Cloud.
• Contribute: Read our Contributing Guide.

---

Reference: Google Cloud Architecture Center - Agentic AI Overview