arc-runtime 0.1.5

Lightweight Python interceptor that prevents AI agent failures in real-time

Arc Runtime

For Team Members: This is the client-side SDK that customers integrate into their applications to enable Arc AI reliability. It intercepts LLM calls, applies learned fixes, and streams telemetry to Arc Core.

What is Arc Runtime?

Arc Runtime is a lightweight Python interceptor that prevents AI agent failures in real-time. It's the client-side component of our Arc AI reliability system - the part that customers install in their applications.

Key Insight: Arc Runtime is invisible to customer applications. It patches LLM SDKs automatically and applies fixes before requests reach the LLM, creating a seamless reliability layer.

---

🚀 Quick Start (Tutorial)

1. Development Setup

git clone https://github.com/arc-computer/runtime.git
cd runtime
pip install -e ".[dev]"

2. Basic Usage

import openai
from runtime import Arc

# Initialize Arc - automatically patches OpenAI
Arc()

# Customer uses OpenAI normally - Arc protects silently
client = openai.OpenAI()
response = client.chat.completions.create(
    model="gpt-4.1",
    messages=[{"role": "user", "content": "Hello"}],
    temperature=0.95  # Arc will fix this to 0.7 automatically
)

3. With Arc Core (Production)

from runtime import Arc

# Connect to Arc Core via Kong Gateway
arc = Arc(
    endpoint="grpc://kong.arc.computer:9080",  # Kong gRPC proxy
    api_key="arc_live_customer_key_here"      # Customer API key
)

# All subsequent LLM calls are protected and telemetry streams to Arc Core

---

📖 How-To Guides

Multi-Agent Pipeline Tracking

from runtime import Arc
import openai

arc = Arc()
client = openai.OpenAI()

# Track complex agent workflows
with arc.create_multiagent_context(application_id="LOAN-2024-001") as ctx:
    # Each agent call is automatically tracked
    response1 = client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role": "user", "content": "Analyze loan application"}],
        extra_headers={"X-Agent-Name": "loan_officer"}
    )
    
    response2 = client.chat.completions.create(
        model="gpt-4.1", 
        messages=[{"role": "user", "content": "Review credit history"}],
        extra_headers={"X-Agent-Name": "credit_analyst"}
    )
    
    # Track context handoffs
    ctx.add_context_handoff(
        from_agent="loan_officer",
        to_agent="credit_analyst", 
        context={"loan_amount": 250000, "assessment": "positive"}
    )
    
    # Get pipeline metrics
    summary = ctx.get_pipeline_summary()

LangGraph Integration

from runtime import ArcStateGraph

# Use ArcStateGraph instead of StateGraph for automatic tracking
workflow = ArcStateGraph()

# Add nodes - each is tracked as an agent
workflow.add_node("process_application", process_fn)
workflow.add_node("verify_documents", verify_fn)

# Compile and run - Arc tracks everything
app = workflow.compile()
result = app.invoke({"application_id": "APP-123"})

Custom Pattern Registration

arc = Arc()

# Register custom failure patterns
arc.register_pattern(
    pattern={"model": "gpt-4.1", "temperature": {">": 0.9}},
    fix={"temperature": 0.7}
)

# Pattern will be applied automatically to matching requests

Testing with Real APIs

# Set API key
export OPENAI_API_KEY="sk-..."

# Run integration tests
python tests/test_real_api.py

# Performance benchmarks
python tests/test_performance.py

---

🔧 Technical Reference

System Architecture

graph TB
    subgraph "Customer Application"
        App[Customer App]
        Arc[Arc Runtime]
        SDK[OpenAI SDK]
        Cache[(Pattern Cache)]
        
        App --> Arc
        Arc --> Cache
        Arc --> SDK
    end
    
    SDK --> API[LLM API]
    API --> SDK
    SDK --> Arc
    Arc --> App
    
    subgraph "Arc Core (Our Infrastructure)"
        Kong[Kong Gateway]
        Core[Arc Core Service]
        DB[(Pattern Database)]
        
        Kong --> Core
        Core --> DB
    end
    
    Arc -.-> Kong
    DB -.-> Cache

Core Components

Component	Purpose	Location
Arc Singleton	Main coordinator, manages interceptor lifecycle	runtime/arc.py
OpenAI Interceptor	Patches OpenAI SDK, applies fixes	runtime/interceptors/openai.py
Pattern Registry	Thread-safe pattern storage and matching	runtime/patterns/
Multi-Agent Context	Pipeline execution tracking	runtime/multiagent/
Telemetry Client	gRPC streaming to Arc Core	runtime/telemetry/
MCP Interceptor	Model Context Protocol monitoring	runtime/interceptors/mcp.py
LangGraph Integration	Automatic workflow tracking	runtime/integrations/langgraph.py

Configuration Options

from runtime import Arc

arc = Arc(
    endpoint="grpc://kong.arc.computer:9080",  # gRPC endpoint
    api_key="arc_live_...",                    # Customer API key
    log_level="DEBUG"                          # Logging level
)

Environment Variables:

• ARC_DISABLE=1 - Disable Arc completely
• ARC_ENDPOINT - gRPC endpoint (default: grpc://localhost:50051)
• ARC_API_KEY - API key for Arc Core
• ARC_LOG_LEVEL - Logging level (default: INFO)

Performance Characteristics

• P99 Interception Overhead: 0.011ms (requirement: <5ms)
• Pattern Matching: O(1) dictionary lookup
• Memory Footprint: <50MB base
• Thread Safety: Full concurrent request support

Metrics & Monitoring

Arc Runtime exposes Prometheus metrics at http://localhost:9090/metrics:

arc_requests_intercepted_total     # Total requests intercepted
arc_fixes_applied_total           # Total fixes applied
arc_pattern_matches_total         # Total pattern matches
arc_interception_latency_ms       # Interception overhead histogram

---

🏗️ Development Workflows

Local Development

# Setup
pip install -e ".[dev]"

# Run tests
pytest
pytest --cov=runtime --cov-report=html

# Code quality
black runtime tests
isort runtime tests
flake8 runtime tests
mypy runtime

Release Process

# Update version in pyproject.toml
# Update CHANGELOG.md
# Run full test suite
pytest

# Build and publish
python -m build
twine upload dist/*

Working with Arc Core

Arc Runtime connects to Arc Core via Kong Gateway:

1. Development: grpc://localhost:50051 (direct Arc Core)
2. Production: grpc://kong.arc.computer:9080 (via Kong)

Kong provides:

• Customer API key authentication
• Rate limiting per customer
• Multi-tenancy isolation
• Load balancing

---

🔍 Troubleshooting

Arc Runtime not intercepting calls

Symptoms: LLM calls bypass Arc, no telemetry generated

Solutions:

1. Import Arc before LLM library:

   from runtime import Arc  # Import Arc first
   Arc()
   import openai  # Then import OpenAI
   

2. Check if disabled:

   echo $ARC_DISABLE  # Should be empty or "0"
   

3. Enable debug logging:

   export ARC_LOG_LEVEL=DEBUG
   

Telemetry not streaming

Symptoms: No data in Arc Core dashboard

Solutions:

1. Check endpoint connectivity:

   telnet kong.arc.computer 9080
   

2. Verify gRPC dependencies:

   pip install grpcio protobuf
   

3. Check API key:

   echo $ARC_API_KEY  # Should start with "arc_live_"
   

Pattern matching not working

Symptoms: Known failure patterns not being fixed

Solutions:

1. Check pattern registry:

   arc = Arc()
   print(arc.get_registered_patterns())
   

2. Verify pattern syntax:

   # Correct
   {"model": "gpt-4.1", "temperature": {">": 0.9}}
   
   # Incorrect
   {"model": "gpt-4.1", "temperature": 0.9}  # Missing operator
   

---

📚 Additional Resources

Team Documentation

• Architecture Decision Records
• Performance Benchmarks
• Testing Strategy

Customer Resources

• Integration Guide
• API Reference
• FAQ

Examples

• Basic Integration
• Multi-Agent Pipeline
• Custom Patterns

---

License

MIT License - see LICENSE for details.