phylax 1.4.0

CI-native regression enforcement for LLM outputs.

Phylax

CI-native regression enforcement for LLM outputs.

---

The Problem

LLM systems change across model versions, prompts, and environments. Phylax enforces contracts so these changes are caught before production.

Installation

pip install phylax

For all LLM providers (OpenAI, Gemini, Groq, Mistral, HuggingFace, Ollama):

pip install phylax[all]

Individual providers:

pip install phylax[openai]       # OpenAI
pip install phylax[google]       # Gemini (google-genai SDK)
pip install phylax[groq]         # Groq LPU
pip install phylax[mistral]      # Mistral AI
pip install phylax[huggingface]  # HuggingFace Inference API
pip install phylax[ollama]       # Ollama (local models)

Quick Start

from phylax import trace, expect, execution, GeminiAdapter

@trace(provider="gemini")
@expect(must_include=["hello"], max_latency_ms=5000)
def greet(name: str):
    """Traced Gemini call with expectations."""
    adapter = GeminiAdapter()
    response, _ = adapter.generate(
        prompt=f"Say hello to {name}",
        model="gemini-2.5-flash",
    )
    return response

# Single call
result = greet("World")
print(result.text)

# Track multi-step agent flows
with execution() as exec_id:
    step1 = greet("Alice")
    step2 = greet("Bob")

# Start the server
phylax server

# Mark a known-good response as baseline
phylax bless <trace_id>

# In CI: fail if output violates declared expectations
phylax check  # exits 1 on contract violation

That's it. Your CI now blocks LLM contract violations.

---

Supported Providers

Provider	Adapter	Env Variable
OpenAI	OpenAIAdapter	OPENAI_API_KEY
Gemini	GeminiAdapter	GOOGLE_API_KEY
Groq	GroqAdapter	GROQ_API_KEY
Mistral	MistralAdapter	MISTRAL_API_KEY
HuggingFace	HuggingFaceAdapter	HF_TOKEN
Ollama	OllamaAdapter	OLLAMA_HOST

from phylax import OpenAIAdapter, GroqAdapter, MistralAdapter

# All adapters share the same interface
adapter = GroqAdapter()
response, trace = adapter.generate(prompt="Hello!", model="llama3-70b-8192")

---

What Phylax is NOT

• ❌ Not monitoring or observability — no metrics, no dashboards, no analytics
• ❌ Not production runtime tooling — CI enforcement only
• ❌ Not AI-based judgment or scoring — rules are deterministic, never LLM-based
• ❌ Not exploratory prompt evaluation — tests outputs against declared contracts
• ❌ Not adaptive or heuristic-driven — exact match, explicit expectations

If you need subjective evaluation or live insights, Phylax is the wrong tool.

---

CI Integration (Primary Interface)

Phylax's primary interface is CI verdict enforcement.

# .github/workflows/phylax.yml
- run: phylax check
  env:
    GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}

Exit codes:

• 0 — All golden traces pass declared expectations
• 1 — Contract violation detected

---

Commands

Command	What it does
phylax init	Initialize config
phylax server	Start API server + UI
phylax list	List traces
phylax list --failed	Show only failed traces
phylax show <id>	Show trace details
phylax replay <id>	Re-run a trace
phylax bless <id>	Mark as golden baseline
phylax check	CI regression enforcement
phylax --version	Show version

---

Capabilities

Capability	Description
Trace Capture	Record every LLM call automatically
Expectations	Validate with @expect rules
Execution Context	Group traces by execution() context
Golden Traces	Baseline comparisons with hash verification
CI Enforcement	phylax check exits 1 on contract violation
Multi-Provider	OpenAI, Gemini, Groq, Mistral, HuggingFace, Ollama
Structured Output Enforcement	JSON field existence, type, value, enum, array bounds
Tool Call Enforcement	Presence, count, argument, and ordering rules
Execution Trace Enforcement	Step count, forbidden transitions, required stages
Cross-Run Stability	Hash comparison and whitelisted drift detection
Metrics & Health	Expectation identity, evaluation ledger, aggregation
Enforcement Modes	enforce / quarantine / observe CI behavior
Meta-Enforcement	Dilution guards: min count, zero signal, change/removal detection

Auxiliary Control Surfaces

The UI and API are auxiliary control surfaces. Phylax's primary interface is CI verdict enforcement.

Surface	Purpose
Web UI	Inspect traces at http://127.0.0.1:8000/ui
Golden Reference UI	Bless/unbless traces from interface
Trace ID Search	Find traces by ID
REST API	Programmatic trace access

---

Architecture

phylax/
├── _internal/           # Core enforcement logic
│   ├── adapters/        # LLM provider adapters
│   ├── expectations/    # Deterministic rule engine (Axis 1)
│   ├── surfaces/        # Surface enforcement layer (Axis 2)
│   ├── metrics/         # Metrics, ledger, health (Axis 3)
│   ├── modes/           # Enforcement modes (Axis 3)
│   ├── meta/            # Meta-enforcement rules (Axis 3)
│   └── graph.py         # Execution graphs
├── cli/                 # CLI commands
├── server/              # API server + health routes
└── ui/                  # Web interface

The API server exists to support Phylax operations (trace storage, golden management, CI verdicts). It is not an extensibility platform.

---

Demos

See the demos/ directory for runnable examples:

python demos/01_basic_trace.py       # Basic tracing
python demos/02_expectations.py      # All @expect rules
python demos/03_execution_context.py # Trace grouping
python demos/04_graph_nodes.py       # Graph API
python demos/05_golden_workflow.py   # CI workflow
python demos/06_raw_evidence.py      # Evidence API
python demos/07_error_contracts.py   # Error codes
python demos/14_metrics_health.py    # Metrics & health reports
python demos/15_enforcement_modes.py # Enforce/quarantine/observe
python demos/16_meta_enforcement.py  # Dilution guards

---

Version

v1.4.0 — Stable Launch: Scale Safety & Misuse Resistance

Expectation engine (Axis 1), Surface Abstraction Layer (Axis 2), and Scale Safety (Axis 3) are stable. 622 tests. 17 enforcement rules. 4 meta-enforcement guards. 3 enforcement modes.

---

Documentation

• Quickstart
• Providers
• Error Codes
• Correct Usage
• API Contract
• Graph Model
• Invariants
• Failure Modes
• Failure Playbook
• Mental Model
• Versioning
• Performance
• Execution Context
• Surface Enforcement
• Axis 2 Testing

---

TL;DR

Phylax is a CI-native, deterministic regression enforcement system for LLM outputs. It records LLM behavior, evaluates explicit expectations, and fails builds when declared contracts regress. Phylax does not explain, score, or optimize outputs — it enforces consistency.

---

License

MIT License

Made with love❤️