cemaf 0.3.0

Context Engineering Multi-Agent Framework - A pluggable, modular framework for building AI agent systems

CEMAF

Context Engineering Multi-Agent Framework

Open source context engineering infrastructure that solves the hard problems in AI agent systems. CEMAF can be used standalone or plugged into existing frameworks like LangGraph, AutoGen, and CrewAI.

---

Table of Contents

• Overview
• The Hard Problems We Solve
• Installation
• Quick Start
• Integration Modes
• Key Features
• Documentation
• Configuration
• Testing
• Contributing
• Getting Help
• Philosophy & Open Startup
• License

---

Overview

CEMAF is a protocol-first framework designed for context engineering in multi-agent AI systems. It provides:

• Token budgeting and automatic context optimization
• Deterministic run recording and replay capabilities
• Full provenance tracking for every context change
• Memory management with strict scoping and TTL
• Zero-config defaults with environment-based customization

Philosophy: Own the hard infrastructure problems while remaining framework-agnostic.

Design: Protocol-based architecture where modules work standalone. Use our defaults or replace them with your own implementations. See Protocol Guide for details.

---

The Hard Problems We Solve

Problem	What Happens	CEMAF Solution
Context Growth	Token limits blow up	Token budgeting + automatic summarization
Reliability	Non-deterministic behavior	Patch-based provenance tracking
Cost	Wasteful token usage	Smart context compilation
Reproducibility	Can't replay/debug runs	Run recording + deterministic replay
Memory Leaks	State bleeds between scopes	Strict memory boundaries with TTL
Content Safety	Harmful outputs slip through	Pre/post-flight moderation gates + PII detection
Quality Drift	Output quality degrades silently	Online eval pipeline with rolling monitors and halt gates
Prompt Engineering	Inconsistent LLM outputs	Semantic blueprints for structured content generation
Spec Drift	Code and intent diverge silently	MetaSpecifier authors OpenSpec proposals; openspec validate --strict is a deterministic eval signal
Zero-to-App	Going from feature idea to runnable code takes days	app_synthesis DAG: description → spec → DAG design → agents → scaffolded, importable CEMAF app on disk
Framework Evolution	Adding new capabilities requires hand-wiring registries, DAGs, bootstrap	Self-hosting meta-layer — CEMAF uses CEMAF to extend CEMAF

---

Installation

# Core installation (minimal dependencies)
pip install cemaf

# With optional integrations
pip install "cemaf[openai]"        # OpenAI + tiktoken
pip install "cemaf[anthropic]"     # Anthropic
pip install "cemaf[tiktoken]"      # Accurate token counting only
pip install "cemaf[prometheus]"    # Prometheus metrics export
pip install "cemaf[all]"           # All optional dependencies

# Development installation
git clone https://github.com/drchinca/cemaf.git
cd cemaf
pip install -e ".[dev]"

Requirements: Python 3.14+

---

Quick Start

from cemaf import Agent, AgentContext, AgentResult, AgentState, AgentRegistry
from cemaf import DAG, Node, create_executor

# 1. Define an agent
class MyAgent(Agent[MyGoal, MyResult]):
    async def run(self, goal, context):
        return AgentResult.ok(output=result, state=AgentState())

# 2. Build a DAG and run it
registry = AgentRegistry()
registry.register_agent(agent_instance=MyAgent(), goal_type=MyGoal)

dag = DAG(name="pipeline", description="My pipeline")
dag = dag.add_node(Node.agent(id="step1", name="Step 1", agent_id="MyAgent", output_key="out"))

executor = create_executor(agent_registry=registry)
result = await executor.run(dag=dag)

See examples/hello_world.py for a complete runnable example.

---

Integration Modes

Mode A: CEMAF Orchestrates

CEMAF owns execution, external frameworks are "engines":

from cemaf.orchestration import DAGExecutor
from cemaf.observability import InMemoryRunLogger

executor = DAGExecutor(
    node_executor=LangGraphNodeExecutor(langgraph_app),
    run_logger=InMemoryRunLogger(),
)
result = await executor.run(dag, context)

# Replay later for debugging
replayer = Replayer(run_logger.get_record("run-123"))
await replayer.replay()

Mode B: CEMAF as Library

External frameworks orchestrate, CEMAF provides infrastructure:

from cemaf.context import Context, ContextPatch
from cemaf.observability import InMemoryRunLogger

@langgraph_node
def my_node(state):
    ctx = Context.from_dict(state)

    # Track provenance of every change
    patch = ContextPatch.from_tool("search", "results", search_results)
    ctx = ctx.apply(patch)
    run_logger.record_patch(patch)

    # Compile within budget
    compiled = compiler.compile(ctx, budget)
    return compiled.to_dict()

See the Integration Guide for detailed patterns.

---

Key Features

Context Engineering

• Context Patches: Track every context change with full provenance
• Token Budgeting: Stay within limits with smart compilation (greedy, knapsack, optimal algorithms)
• Deterministic Replay: Record and replay runs for debugging
• Glass Box Audit: Full provenance chain linking every LLM call to its context sources, citations, and costs
• Context Type Classification: RESOURCE/MEMORY/SKILL behavioral semantics with per-type compaction rules
• Semantic Blueprints: Structured content generation with Denis Rothman's blueprint pattern
• Recursive LLM: Parallel divide-and-conquer querying for 1M+ token contexts

Memory System

• Strict Scoping: Memory boundaries with TTL prevent state leaks
• Three-Tier Progressive Loading: L0 abstract / L1 overview / L2 full content for token-efficient retrieval
• Semantic Deduplication: Exact key + embedding similarity detection with merge/skip resolution
• Post-Session Extraction: Automatic promotion of session learnings to long-term memory (patterns, corrections, facts)
• Hierarchical Scope Propagation: Parent-to-child score propagation for scope-aware retrieval
• SQLite Persistence: Production-ready persistent memory store via aiosqlite

Online Evaluation

• Hierarchical Judge: Three-tier evaluation -- fast deterministic checks, semantic similarity, LLM judge (escalates only when needed)
• Online Eval Pipeline: Subscribe to execution events and run evaluators on node outputs in real-time
• Quality Police: Rolling window quality monitor with anomaly detection and automatic halt gates
• Eval Tools & Agents: RunEvalTool, CheckQualityTool, RecordScoreTool, QualityGuardAgent -- dogfooding the eval system as CEMAF tools

Production Backends

• Resilient LLM Client: Retry with exponential backoff + circuit breaker + rate limiter composing around any LLMClient
• OpenAI Embeddings: Production embedding provider using text-embedding-3-small with batch support
• Structured Logging: JSON-lines logger with context fields for production observability
• Prometheus Metrics: Counter/gauge/histogram/timing export with lazy metric registration

Orchestration

• DAG Executor: Topological sort, parallel execution, conditional routing, loop nodes, cooperative cancellation
• Node Type Handlers: Extracted router, conditional, loop, parallel handlers for clean separation
• RuntimeServices: Frozen dataclass bundling 15+ optional dependencies for composition root
• Bootstrap: Single create_executor() entry point wiring registry, services, and subscriptions
• Context Agents: Built-in Librarian, Researcher, Summarizer, Writer agents with dynamic registry
• Budget Guard: Configurable cost and token limits with warning/critical/halt thresholds

Infrastructure

• Protocol-Based: Plug into any framework -- modules work standalone, extend with your own implementations
• Extensible Registries: ProviderRegistry pattern for pluggable LLM, compiler, and retrieval backends
• Instrumented LLM: Transparent LLM call recording for automatic glass box audit
• Moderation & Guardrails: Pre/post-flight content safety with PII detection and compliance rules
• Configuration-Driven: Zero-config defaults with .env customization
• Resilience: Retry, circuit breaker, rate limiting as composable decorators

Self-Hosting Engine

CEMAF is its own first client — opt-in modules where the framework uses its own primitives to introspect, audit, spec, and extend itself. Fully decoupled from the base framework (one-way dependency).

• Audit Trail: EventBusAuditLog subscribes to EventBus, converts events into queryable AuditEntry records with quality trend analysis and z-score anomaly detection
• Knowledge Graph: MemoryBackedKnowledgeGraph — entities and relations backed by MemoryManager with semantic search and neighbor traversal
• Meta-Agents: MetaArchitect (DAG design), MetaSpecifier (OpenSpec proposal authoring), MetaSynthesizer (agent code gen), MetaAuditor (trace analysis), MetaKnowledgeGraph (KG operations), MetaScaffolder (runnable CEMAF-app synthesis)
• OpenSpec Bridge: OpenSpecRuntime protocol (System/Npx/Fake impls) + OpenSpecWorkspace (atomic writes, per-change locks) exposes openspec validate/list/show/write/delete as CEMAF tools
• Pre-built DAGs: create_self_audit_dag(), create_feature_synthesis_dag(), create_knowledge_refresh_dag(), create_self_spec_dag(), create_app_synthesis_dag()
• Entry point: create_meta_executor() wraps create_executor(), auto-wires audit + KG from RuntimeServices and MetaSpecifier/OpenSpec tools from MetaServices

What this gets you: one instruction ("build an app that does X") becomes a working CEMAF-based app on disk — spec validated by openspec validate --strict, agents synthesized from the spec, scaffolded into an importable package with its own registry, DAGs, and smoke tests. See create_app_synthesis_dag().

---

Documentation

Full Documentation →

Getting Started

• Quick Start Guide - Get running in 5 minutes
• Protocol Guide - Understanding CEMAF's protocol-based architecture
• Extension Patterns - How to extend CEMAF with your own implementations
• Standalone Usage - Using modules independently

Core Guides

• Architecture Overview - System design and principles
• Context Management - Patches, provenance, budgeting
• Replay & Recording - Deterministic replay
• Tools, Skills, Agents - Execution layer
• Integration Guide - Framework integration patterns

Module References

• LLM Integration
• Caching
• Persistence
• Observability
• Citation Tracking - Source attribution
• MCP Integration - Model Context Protocol
• Blueprint - Semantic blueprints for content generation
• Moderation - Guardrails and content safety
• Retrieval - Vector stores and search

---

Configuration

CEMAF is designed for zero-config startup with production-ready defaults. Customize via environment variables:

# Copy example configuration
cp .env.example .env

# Configure your setup
CEMAF_LLM_PROVIDER=openai
CEMAF_LLM_API_KEY=your-key
CEMAF_CACHE_BACKEND=redis
CEMAF_CACHE_MAX_SIZE=10000

Use factory functions for automatic configuration loading:

from cemaf.llm import create_llm_client_from_config
from cemaf.cache import create_cache_from_config

# Automatically loads from .env or environment
client = create_llm_client_from_config()
cache = create_cache_from_config()

See the Configuration Guide for all available settings.

---

Testing

# Run all tests
pytest tests/

# Unit tests only
pytest tests/unit/

# Skip slow tests
pytest tests/ -m "not slow"

# With coverage
pytest tests/ --cov=cemaf

# Pre-commit checks
pre-commit run --all-files

Project Stats: 2301+ tests | 100% passing | TDD from day one

---

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Development setup:

# Fork and clone the repo
git clone https://github.com/YOUR_USERNAME/cemaf.git
cd cemaf

# Install dependencies with uv
uv venv
uv sync

# Install pre-commit hooks
uv run pre-commit install

See HOW_TO_USE.md for detailed usage examples.

---

Getting Help

We're here to help! Here are the best ways to get support:

Documentation

• Full Documentation - Comprehensive guides for all features
• Quick Start Guide - Get started in minutes
• HOW_TO_USE.md - Detailed usage patterns
• Architecture Guide - Understand CEMAF's design

Community

• Discord Server - Join our community for real-time help
• GitHub Discussions - Ask questions and share ideas
• GitHub Issues - Report bugs or request features

Contributing

Want to contribute? Check out our Contributing Guide to get started!

We're in Alpha and actively seeking feedback!

---

Philosophy & Open Startup

CEMAF operates as an open startup - we believe in radical transparency, community collaboration, and building in public.

Our Principles

• Community First: We serve developers building AI agents
• Transparent: All decisions, metrics, and roadmap are public
• Bias Toward Action: Show > tell. Open PRs, not long debates
• Anyone Can Help: Contribution > credentials
• Learn in Public: We share wins AND mistakes

Resources

• Philosophy Guide - Our 10 core principles and values
• Open Metrics - Transparent metrics, roadmap, and financials
• Decision Log - All major decisions documented
• Weekly Updates - Progress, learnings, and challenges

We're building CEMAF together. Your voice matters.

---

License

This project is licensed under the MIT License - see the LICENSE file for details.

---

Authors

Hikuri Bado Chinca (@drchinca) Email: [e-mail]

Copyright (c) 2026 | Published on 1.1.2026 🎉

---

Links

• Documentation: docs/README.md
• Issues: GitHub Issues
• Contributing: CONTRIBUTING.md