phylax 1.0.1

Deterministic regression enforcement for LLM systems.

Phylax

Deterministic regression enforcement for LLM systems.

---

The Problem

LLM outputs change unexpectedly. Same prompt, different model version → different behavior. Without Phylax, you discover this in production.

Installation

pip install phylax

For server/UI support:

pip install phylax[server]

For all LLM providers:

pip install phylax[all]

Quick Start

from phylax import trace, expect, execution

@trace(provider="gemini")
@expect(must_include=["refund"], max_latency_ms=1500)
def customer_reply(query):
    return llm.generate(query)

# Track multi-step agent flows
with execution("customer-support-flow"):
    result = customer_reply("I want a refund")

# Mark a known-good response as baseline
phylax bless <trace_id>

# In CI: fail if output regresses
phylax check  # exits 1 on failure

That's it. Your CI now blocks LLM regressions.

---

What Phylax is NOT

• ❌ Not monitoring — no metrics, no dashboards
• ❌ Not observability — no traces-to-cloud, no analytics
• ❌ Not AI judgment — rules are deterministic, not LLM-based
• ❌ Not cloud-dependent — runs entirely local
• ❌ Not prompt engineering — tests outputs, not prompts

Phylax is a test framework. It tells you when LLM behavior changes.

---

CI Integration

# .github/workflows/phylax.yml
- run: phylax check
  env:
    GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}

Exit codes:

• 0 — All golden traces pass
• 1 — Regression detected

---

Expectations (Deterministic Rules)

@expect(
    must_include=["word"],       # Required content
    must_not_include=["sorry"],  # Forbidden content
    max_latency_ms=2000,         # Performance gate
    min_tokens=10                # Minimum length
)

All rules are deterministic. No AI judgment. No ambiguity.

---

Commands

Command	What it does
phylax init	Initialize config
phylax server	Start API server
phylax list	List traces
phylax list --failed	Show only failed traces
phylax show <id>	Show trace details
phylax replay <id>	Re-run a trace
phylax bless <id>	Mark as golden baseline
phylax check	CI regression check

---

Features

Feature	Description
Trace Capture	Record every LLM call automatically
Expectations	Define PASS/FAIL rules (4 deterministic rules)
Golden Traces	Baseline comparisons with hash verification
CI Integration	phylax check exits 1 on regression
Execution Graphs	Visualize multi-step agent workflows
Forensics Mode	Debug failures with guided investigation

---

Stability Guarantee

Phylax v1.0.0 is API-frozen:

• No breaking changes in v1.x
• trace, expect, execution are stable
• Exit codes are stable
• Schema is stable

See docs/contract.md for full guarantees.

---

Documentation

• Quickstart
• Mental Model
• API Contract

---

License

MIT License