electripy-studio 0.3.1

Production-grade Python toolkit for AI product engineering — LLM gateway, policy guardrails, RAG eval, agent collaboration, telemetry, and more.

ElectriPy Studio

Production-minded Python components and recipes (cookbook) by Inference Stack.

Overview

ElectriPy Studio is a curated collection of production-ready Python components and recipes designed to accelerate development while maintaining high code quality standards.

Design principles

• Ports & Adapters: swap providers (LLMs, embedders, vector stores) without rewriting business logic.
• Deterministic by default: stable IDs and reproducible evaluation runs.
• Safe logging posture: avoid leaking prompts/responses; prefer hashes + redaction seams.
• Typed, production APIs: small public surfaces, strict typing, structured outputs where it matters.
• Testability: unit tests are offline and deterministic by default (no network required).

How ElectriPy compares

ElectriPy is not a framework — it's a composable toolkit of production-grade building blocks for AI-powered Python applications. Here's how it relates to popular alternatives:

Library	Stars	Overlap	ElectriPy's edge
LiteLLM	40 k	Provider-agnostic LLM gateway	Bundles policy hooks, telemetry & structured output inline — no proxy server needed
Guardrails AI	6.6 k	Input / output validation	Lighter-weight, composable policy gateway — no XML DSL or Hub dependency
CrewAI / AutoGen	50 k+	Multi-agent orchestration	Bounded & deterministic with hop limits; building-block, not a framework
RAGAS	13 k	RAG evaluation metrics	Integrates eval directly into CLI & CI gating; ships with drift comparison
Instructor	12.6 k	Structured LLM output	Dedicated Structured Output Engine with retry + temperature decay, plus caching & replay tape
Pydantic AI	10 k+	Typed AI agents	Narrower scope; ElectriPy ships caching, replay tape, eval assertions, multi-provider adapters & more
Haystack / LangChain	40 k+	Full RAG / agent framework	Composable building blocks you import — not a framework you adopt wholesale

TL;DR — Use ElectriPy when you want discrete, well-typed utilities that compose into your architecture rather than a monolithic framework that owns it.

Status & recent updates

• Last updated: 2026-03-25
• Maturity: Early alpha (APIs may still evolve), but core components, CLI, concurrency primitives, and a growing suite of AI product engineering utilities are in place.
• Versioning: SemVer begins at v0.x — expect breaking changes until v1.0.
• Recent highlights:
  • Added Fallback Chain — automatic provider failover across ranked SyncLlmPort adapters with metadata tracking.
  • Added Batch Complete — fan-out N LLM requests with bounded concurrency, order-preserving results, and per-request error isolation.
  • Added Cost Ledger — thread-safe token cost accumulation with per-label slicing (tenant, model, feature).
  • Added Prompt Fingerprint — deterministic SHA-256 request hashing for caching, dedup, and audit trails.
  • Added JSON Repair — fix markdown fences, trailing commas, single quotes, unquoted keys, mismatched brackets, and truncated JSON in one call.
  • Added Circuit Breaker — closed→open→half_open FSM protecting against cascading provider failures.
  • Added Sensitive Data Scanner — regex-based PII and secret detection (email, phone, SSN, API keys, AWS keys) with extensible patterns.
  • Added a Structured Output Engine — extract typed Pydantic models from LLM text with auto-retry and temperature decay.
  • Added an LLM Caching Layer — pluggable response caching (in-memory LRU, SQLite WAL) with hit-rate tracking.
  • Added an LLM Replay Tape — record, replay, and diff LLM interactions for deterministic offline tests.
  • Added Eval Assertions — pytest-native assertion helpers for LLM outputs with structured diagnostics.
  • Added Provider Adapters — Anthropic (Messages API) and Ollama (HTTP) adapters for multi-provider support.
  • Added electripy demo policy-collab CLI command — run the full policy + agent collaboration pipeline offline with a Rich table report.
  • Added a Policy Gateway with regex-based detection, sanitization, deny/allow/require-approval actions across preflight, postflight, stream, and tool-call stages.
  • Added a Agent Collaboration Runtime for bounded multi-agent orchestration with hop limits, deque-based message routing, and optional policy gateway integration.
  • Added LLM Gateway request/response hooks and build_llm_policy_hooks() bridge so policy decisions plug directly into the LLM call path.
  • Phase 1–2: LLM Gateway, RAG eval, streaming chat, agent runtime, hallucination guard, response robustness, prompt engine, token budget, context assembly, model routing, conversation memory, tool registry, AI telemetry.
  • Expanded documentation and user guides for all components.

Features

• 🔧 Core Components: Configuration, logging, error handling, and type utilities
• ⚡ Concurrency: Retry mechanisms (sync/async), async token bucket rate limiter, and circuit breaker for cascading failure protection
• 📁 I/O: JSONL read/write utilities for efficient data processing
• 💻 CLI: Typer-based command-line interface with health checks, RAG eval runner, and an offline demo showcase (electripy demo policy-collab)
• 🤖 AI building blocks: Provider-agnostic LLM Gateway with sync/async clients, request/response policy hooks, structured-output helpers, and a RAG Evaluation Runner for retrieval benchmarking.
• � Multi-provider adapters: Anthropic (Messages API) and Ollama (HTTP) adapters plus OpenAI and generic HTTP-JSON — swap providers with a one-line change.
• 🎯 Structured Output Engine: Parse LLM text into typed Pydantic models with auto-retry and temperature decay for convergence.
• 💾 LLM Caching Layer: Pluggable response caching (in-memory LRU, SQLite WAL) with deterministic cache keys, hit-rate tracking, and custom backend support.
• 🔄 LLM Replay Tape: Record, replay, and diff LLM interactions — JSONL-serializable tapes for deterministic offline tests and output regression detection.
• ✅ Eval Assertions: pytest-native assertion helpers (keyword, regex, JSON schema, predicate, length) with structured diagnostic reports for CI.
• �📊 AI Telemetry: Provider-agnostic telemetry primitives and adapters (JSONL, optional OpenTelemetry) for HTTP resilience, LLM gateway, policy decisions, and RAG evaluation runs.
• 🧠 AI product engineering utilities: Streaming chat primitives, deterministic agent runtime helpers, RAG quality/drift metrics, grounding checks for hallucination reduction, response robustness helpers for structured outputs, prompt templating and composition, token budget tracking and truncation, priority-based context window assembly, rule-based model routing, sliding-window conversation memory, and a declarative tool registry with JSON schema generation.
• 🛡️ AI policy and collaboration runtime: Deterministic policy gateway checks for preflight/postflight/stream/tool flows, plus bounded agent-to-agent collaboration runtime for specialist orchestration patterns.
• 🔗 Fallback Chain: Automatic provider failover — tries ranked LLM adapters in order with metadata tracking.
• 📦 Batch Complete: Fan-out N LLM requests with bounded concurrency, order-preserving results, and per-request error isolation.
• 💰 Cost Ledger: Thread-safe token cost accumulation with multi-dimensional label slicing.
• 🔑 Prompt Fingerprint: Deterministic SHA-256 request hashing for caching, dedup, and drift detection.
• 🔧 JSON Repair: Fix 7 common LLM JSON breakage patterns (fences, trailing commas, single quotes, unquoted keys, mismatched brackets, truncation) in one call.
• 🔒 Sensitive Data Scanner: Regex-based PII and secret detection with 9 built-in patterns and extensible custom rules.

Quick Start

Using ElectriPy as a library

For most users, ElectriPy is just a Python library you depend on in your own project.

Install from PyPI:

pip install electripy-studio

To work against a local clone in editable mode (e.g., to experiment with changes while using it in another project):

pip install -e .

For development on this repo itself (full tooling and test extras):

pip install -e ".[dev]"

Verify Installation

electripy doctor

Basic Usage

from electripy import Config, get_logger
from electripy.concurrency import retry, AsyncTokenBucketRateLimiter
from electripy.io import read_jsonl, write_jsonl

# Configuration
config = Config.from_env()
logger = get_logger(__name__)

# Retry with exponential backoff
@retry(max_attempts=3, delay=1.0, backoff=2.0)
def fetch_data():
    return api_call()

# Rate limiting
limiter = AsyncTokenBucketRateLimiter(rate=10, capacity=10)
async with limiter:
    await rate_limited_operation()

# JSONL I/O
data = [{"id": 1, "name": "Alice"}, {"id": 2, "name": "Bob"}]
write_jsonl("output.jsonl", data)

for record in read_jsonl("output.jsonl"):
    print(record)

AI quick start (LLM + RAG eval)

• Run a basic RAG evaluation over JSONL datasets:

electripy rag eval --corpus data/corpus.jsonl --queries data/queries.jsonl \
    --top-k 3,5,10 --report-json out/report.json

• LLM Gateway usage (offline-friendly fake provider example): see recipes/02_llm_gateway/.

Demo: Policy Gateway + Agent Collaboration

Run the full pipeline offline — no API keys needed:

electripy demo policy-collab

Customise with --prompt and --max-hops:

electripy demo policy-collab --prompt "Alert [e-mail] about outage" --max-hops 6

See recipes/03_policy_collaboration/ for the standalone script.

Documentation

Full documentation is available in the docs/ directory:

• Installation Guide
• Quickstart
• Core Concepts
• Concurrency & Resilience
• I/O Utilities
• CLI Guide
• LLM Gateway & AI
• AI Telemetry
• AI Policy Gateway
• AI Agent Collaboration Runtime
• Structured Output Engine
• LLM Caching Layer
• LLM Replay Tape
• Eval Assertions
• Provider Adapters (Anthropic, Ollama)
• Fallback Chain
• Batch Complete
• Cost Ledger
• Prompt Fingerprint
• JSON Repair
• Sensitive Data Scanner
• Circuit Breaker
• RAG Evaluation Runner
• AI Product Engineering Utilities
• Component Maturity Model
• Recipes
• API Reference

Build and serve docs locally:

pip install -e ".[docs]"
mkdocs serve

Visual Overview

Repository Map

Development Workflow

Project Structure

electripy-studio/
├── src/electripy/          # Main package
│   ├── core/               # Config, logging, errors, typing
│   ├── concurrency/        # Retry, rate limiting & circuit breaker
│   ├── io/                 # JSONL utilities
│   ├── cli/                # CLI commands
│   └── ai/                 # AI building blocks and product-engineering utilities
│       ├── llm_gateway/    # Provider-agnostic LLM client + structured output helpers
│       ├── rag_eval_runner/# Dataset + eval runner + CLI benchmarking
│       ├── streaming_chat/ # Sync/async stream chunk and collection helpers
│       ├── agent_runtime/  # Deterministic tool-plan execution primitives
│       ├── rag_quality/    # Retrieval metrics and drift comparison helpers
│       ├── hallucination_guard/ # Grounding and citation checks
│       ├── response_robustness/ # JSON extraction/repair and output guards
│       ├── prompt_engine/       # Template composition and few-shot management
│       ├── token_budget/        # Pluggable token counting and truncation
│       ├── context_assembly/    # Priority-based context window packing
│       ├── model_router/        # Rule-based model selection and routing
│       ├── conversation_memory/ # Sliding window and token-aware chat history
│       ├── structured_output/   # Pydantic extraction with retry & temp decay
│       ├── llm_cache/           # Pluggable response caching (memory, SQLite)
│       ├── replay_tape/         # Record, replay, and diff LLM interactions
│       ├── eval_assertions/     # pytest-native assertion helpers for LLM outputs
│       ├── policy_gateway/      # Deterministic pre/post/tool/stream policy decisions
│       ├── tool_registry/       # Declarative tool definitions and JSON schema
│       ├── agent_collaboration/ # Bounded multi-agent handoff orchestration
│       ├── fallback_chain.py   # Automatic provider failover
│       ├── batch_complete.py   # Concurrent LLM fan-out with backpressure
│       ├── cost_ledger.py      # Thread-safe token cost accumulation
│       ├── prompt_fingerprint.py # Deterministic SHA-256 request hashing
│       ├── json_repair.py      # Fix common LLM JSON breakage
│       └── sensitive_data_scanner.py # PII & secret detection
├── tests/                  # Test suite
├── docs/                   # Documentation
├── recipes/                # Example recipes
│   ├── 01_cli_tool/        # CLI tool example
│   ├── 02_llm_gateway/     # LLM gateway examples
│   └── 03_policy_collaboration/ # End-to-end policy + multi-agent flow
├── packages/               # NPM packages
│   └── electripy-cli/      # NPM CLI wrapper
├── pyproject.toml          # Project config
├── mkdocs.yml              # Docs config
└── LICENSE                 # MIT License

Development

Running Tests

pytest tests/ -v

With coverage:

pytest tests/ -v --cov=src --cov-report=term-missing

Code Quality

# Linting
ruff check .

# Formatting
black .

# Type checking
mypy src/

Python Tooling (recommended)

These tools are optional but recommended for contributors working on ElectriPy Studio itself. They are installed globally (via pipx) and then used inside whatever project or virtualenv you prefer.

1. Install global CLI tools with pipx

pipx lets you install Python CLIs in isolated environments, so they don't conflict with your project dependencies:

python -m pip install --upgrade pip

brew install pipx      # or see https://pipx.pypa.io for other platforms
pipx ensurepath

pipx install uv        # fast Python package/dependency manager
pipx install poetry    # project/virtualenv manager (optional; this repo uses pyproject + hatchling)
pipx install ruff      # fast linter (also available via .[dev] extra)
pipx install pre-commit  # git pre-commit hooks runner

2. Using uv (optional)

uv is a fast drop-in for many pip/python -m venv workflows. For example, to create a fresh environment for hacking on ElectriPy Studio:

uv venv .venv
source .venv/bin/activate

uv pip install -e ".[dev]"

You can also use uv pip install electripy-studio in your own projects.

3. Using poetry in your own projects (optional)

This repo is built with pyproject.toml + Hatchling, but you can happily consume ElectriPy from a Poetry-managed project:

poetry add electripy-studio

The library itself has no dependency on Poetry; it's just a convenient project manager if you already use it.

4. pre-commit (for contributors)

Once pre-commit is installed, enable the hooks defined in .pre-commit-config.yaml:

pre-commit install

This will automatically run Black, Ruff, and basic whitespace checks on changed files before each commit.

CI/CD

GitHub Actions automatically runs tests, linting, and type checking on all pull requests. See .github/workflows/ci.yml.

Recipes

Check out the recipes/ directory for complete examples:

• 01_cli_tool — Building a production-ready CLI tool
• 02_llm_gateway — LLM Gateway basics using a fake provider (offline-friendly)
• 03_policy_collaboration — End-to-end policy gateway + LLM hooks + multi-agent collaboration demo

Additional recipe guides are available in the docs:

• Policy Gateway recipe — standalone policy evaluation walkthrough
• Agent Collaboration Runtime recipe — bounded agent handoff patterns
• Policy + Collaboration E2E recipe — full pipeline with telemetry
• RAG Evaluation Runner recipe — benchmarking retrieval quality
• AI Telemetry recipe — wiring observability across components

Requirements

• Python 3.11 or higher
• Dependencies managed via pyproject.toml

License

MIT License - See LICENSE for details.

Contributing

Contributions are welcome! Please read our Contributing Guide and Code of Conduct before submitting PRs. For security issues, see SECURITY.md.

Links

• GitHub Repository
• Documentation
• Issue Tracker