Epires — Epistemic Auto-Research Harness

версия на русском доступна в README_RU.md.

📑 Table of Contents

1. Introduction: The Autoresearch Dilemma
2. Quickstart & Installation
3. Architecture Overview
4. Research Methodology & Epistemology
   • 4.1 VSA Hypergraph: Experiments as Hypervectors
   • 4.2 Hypothesis-First & Popperian Falsification
   • 4.3 Evidence Scaling (E0–E5) and Source Provenance [V]/[P]/[D]
   • 4.4 Cascading DAG Invalidation
   • 4.5 Separation of Concerns: Lead-PI vs Coder Subagents
   • 4.6 Zero-Overhead Epistemic Tracing
   • 4.7 Antifragile Dual-Mode Onboarding
5. What's New in 0.4.0: Verifiable Epistemics
6. CLI Command Reference
7. Model Context Protocol (MCP) Specification
8. Eval Sandbox
9. Testing & Mathematical Fuzzing
10. License

1. Introduction: The Autoresearch Dilemma

Current automated research and coding agents (e.g., vanilla Autoresearch, AIDE, naive Keep-or-Revert loops) typically follow a greedy linear search loop:

$$ \text{Code} \xrightarrow{\text{LLM Mutation}} \text{New Code} \xrightarrow{\text{Eval Script}} \text{Scalar Metric} \xrightarrow{\text{Keep or Revert}} \dots $$

When applied to serious scientific ML, quantitative finance, or complex systems, this paradigm suffers from critical failure modes:

1. Reward Hacking & Horizon Truncation: Agents overfit to local validation windows, cherry-pick random seeds, and introduce technical debt to maximize a transient scalar.
2. Loss of Epistemic Memory: The system fails to understand why a mutation succeeded and discards the valuable search-space reduction provided by negative results.
3. Flat Code Mutations vs Causal DAGs: Experiments are treated as isolated text diffs rather than a structured causal graph of interconnected theoretical hypotheses.

Epires addresses this by providing an end-to-end auto-research harness integrating Vector Symbolic Architectures (VSA), Hypothesis Dependency DAGs, and automated epistemic gatekeeping.

2. Quickstart & Installation

Installation via pip

Install from PyPI:

pip install epires

Or install directly from source for local development:

git clone https://github.com/himera/epires.git
cd epires

# Editable installation with dev dependencies
pip install -e ".[dev]"

(For uv users: uv add epires or uv sync --extra dev)

Workspace Initialization

The init command performs a safe, non-destructive setup in the current directory (supporting both clean repositories and existing codebases):

# 1. Reconnaissance scan (detects domain, existing docs, stack, and metrics)
epires recon

# 2. Initialize research environment (.epires/, .gitignore, config.json, MCP config)
epires init

Running the Services

# 1. Start the MCP Server for AI coding assistants (Cursor, Claude Code, Antigravity):
epires mcp

# 2. Start the FastAPI REST API Server & Web Dashboard (open http://localhost:8000 in your browser):
epires serve --port 8000

# 3. Inspect research status and generate Mermaid DAG in the terminal:
epires status
epires dag

3. Architecture Overview

┌────────────────────────────────────────────────────────────────────────┐
│                        LLM Agent / Subagents                           │
│   (Governed by Cognitive Protocol: skills/epires_researcher)           │
└───────────────────┬────────────────────────────────┬───────────────────┘
                    │ Model Context Protocol (MCP)   │ Real-time Delta Stream
                    ▼                                ▼
┌────────────────────────────────────────────────────────────────────────┐
│                    Epires Core & Server Engine                         │
│                                                                        │
│  ┌──────────────────────┐  ┌────────────────────┐  ┌────────────────┐  │
│  │   VSA Hypergraph     │  │ Cascading DAG      │  │ Auto-Tracer    │  │
│  │   (10,000-D Engine)  │  │ (Falsification)    │  │ (SQLite + MD)  │  │
│  └──────────┬───────────┘  └─────────┬──────────┘  └────────┬───────┘  │
└─────────────┼────────────────────────┼──────────────────────┼──────────┘
              │                        │                      │
              ▼                        ▼                      ▼
┌─────────────────────────┐  ┌───────────────────┐  ┌───────────────────┐
│   .epires/hypotheses.db │  │ docs/agent-trace  │  │ Research Atlas UI │
│   (SQLite + VSA Vectors)│  │ (Audit Trail)     │  │ (Web Dashboard)   │
└─────────────────────────┘  └───────────────────┘  └───────────────────┘

Module Layout:

• epires_core/vsa.py — 10,000-dimensional Bipolar VSA algebra ($\text{bind}$, $\text{permute}$, $\text{bundle}$, cosine similarity).
• epires_core/hypergraph.py — Vectorization of n-ary experimental hyperedges into hypervectors.
• epires_core/store.py — Embedded SQLite storage, vector index, and cascading DAG invalidation.
• epires_core/tracer.py — Automated logger syncing actions between SQLite and docs/agent-trace.md.
• epires_core/config.py — Dynamic project configuration (.epires/config.json) and heuristic topology scanner.
• tools/web_search.py — Multi-query literature and web search gateway powered by parallel-web 1.3.0 SDK.
• server/app.py — FastAPI REST API & Web Dashboard backend (CRUD, Gap Analysis, Stratigraphy, Provenance, WebSockets).
• server/static/ — Research Atlas SPA frontend (interactive DAG visualizer, dossier, timeline, artifacts explorer).
• server/mcp_server.py — FastMCP server exposing 28 research tools to AI agents.
• skills/epires_researcher/SKILL.md — Cognitive protocol and operational standard for the Lead Principal Investigator.

🌐 Research Atlas (Web Dashboard)

Epires includes a built-in real-time observational research atlas accessible at http://localhost:8000:

• Live Reactive DAG Visualizer: Interactive Voronoi pebble cards with Catmull-Rom cubic splines, smart multi-subgraph matrix layout, node dragging, and automatic position persistence.
• Monograph Dossier: Comprehensive 5-section inspection panel featuring theoretical mechanisms, Popperian falsification thresholds, metric delta confidence intervals (CI95), and entity combinations.
• Stratigraphy Event Stream: Complete chronological timeline uniting hypothesis registrations, empirical evidence verdicts, and agent trace entries.
• Entity Coverage Grid: Matrix projection across Cartesian spaces ($\text{Model} \times \text{Feature} \times \text{Regime}$) to instantly spotlight unexplored white spaces.
• Provenance Ledger & Artifacts Explorer: Transparent audit trail mapping citations, code commits, evidence claims, and local filesystem artifacts (artifacts/) with direct file viewing.
• Zero-Latency WebSocket Sync: Instant delta broadcast (/ws) reflecting agent actions in real time with ultra-low polling fallback (/atlas/version).
• Design Aesthetic: Swiss coordinate dot matrix canvas with Bayer ordered dithering and risograph halftone shaders.

4. Research Methodology & Epistemology

[Web/ArXiv Search] ➔ [VSA Gap Discovery] ➔ [Register Hypothesis] ➔ [Contract Delegation] ➔ [Zero-Trust Audit] ➔ [Log Evidence & DAG Update] ➔ [AutoTrace]

4.1 VSA Hypergraph: Experiments as Hypervectors

In traditional graph databases (Neo4j, RDF), an experiment is fractured across dozens of binary relations $(A \to B)$, losing structural integrity. Epires implements Hypergraph-as-a-Vector (inspired by HSME and Kanerva SDM memory architectures):

Each hypothesis or experiment is encoded as an n-ary hyperedge into a single bipolar hypervector $\mathbf{v} \in {-1, +1}^D$ ($D = 10{,}000$) using three algebraic operations:

1. Bind ($\otimes$): Element-wise multiplication binding roles to values:

$$ \mathbf{v}{\text{bound}} = \mathbf{v}{\text{role}} \odot \mathbf{v}_{\text{value}} $$

Property: Reversible ($\mathbf{v}{\text{bound}} \odot \mathbf{v}{\text{role}} = \mathbf{v}_{\text{value}}$) and quasi-orthogonal to both inputs.

2. Permute ($\sigma$): Cyclic vector permutation encoding relation directionality:

$$ \mathbf{v}{\text{edge}} = \mathbf{v}{\text{src}} \odot \mathbf{v}{\text{rel}} \odot \sigma^k(\mathbf{v}{\text{tgt}}) $$

3. Bundle ($\oplus$): Majority voting bundling multiple entities and relations into a single composite representation:

$$ \mathbf{v}{\text{hyperedge}} = \mathrm{sign}\left(\sum{i=1}^M \mathbf{v}_i\right) $$

Automated Gap Discovery

Using algebraic projections in the hypervector space, Epires computes unstudied Cartesian product combinations across dimensions:

$$ \text{Gaps} = (\mathcal{M}{\text{models}} \times \mathcal{F}{\text{features}} \times \mathcal{R}{\text{regimes}}) \setminus \mathcal{E}{\text{tested}} $$

4.2 Hypothesis-First & Popperian Falsification

«No execution without prior mathematical justification and numerical falsification criteria.»

Every hypothesis must be registered in the VSA Hypergraph before code implementation begins, specifying:

1. A Priori Mechanism: The theoretical rationale establishing why the hypothesis should hold.
2. Popperian Falsification Criteria: Explicit numerical boundaries that refute the claim (e.g., «RMSLE on OOT validation exceeds 1.85» or «SDM hit@1 precision < exact kNN by > 5%»).

4.3 Evidence Scaling (E0–E5) and Source Provenance [V]/[P]/[D]

Claims must never exceed their verified evidence level:

Source Provenance Tags:

• [V] (Primary Verified) — Directly verified primary artifact or source code;
• [P] (Secondary Reported) — External report or leaderboard submission;
• [D] (Inferred) — Deduced from adjacent literature.

4.4 Cascading DAG Invalidation

Hypotheses are structured in a directed acyclic graph (DEPENDS_ON).

graph TD
  classDef confirmed fill:#2ea043,stroke:#1b4b27,color:#fff;
  classDef falsified fill:#da3633,stroke:#8e1519,color:#fff;
  classDef blocked fill:#6e7681,stroke:#30363d,color:#fff;
  classDef in_prog fill:#d29922,stroke:#bb8009,color:#fff;

  H1["H1: SDM Memory<br/>[FALSIFIED]"]:::falsified
  H2["H2: Adaptive Read/Write<br/>[BLOCKED]"]:::blocked
  H3["H3: Execution Router<br/>[BLOCKED]"]:::blocked
  H4["H4: Direct Baseline<br/>[CONFIRMED]"]:::confirmed
  H5["H5: Wavelet Features<br/>[IN_PROGRESS]"]:::in_prog

  H2 -->|depends_on| H1
  H3 -->|depends_on| H2
  H5 -->|depends_on| H4
  H1 -.->|BLOCKS| H2
  H1 -.->|BLOCKS| H3

When an empirical experiment falsifies parent hypothesis $H_1$, the _cascade_falsification engine computes the transitive closure and automatically transitions all downstream dependent hypotheses ($H_2, H_3$) into BLOCKED status, preventing wasted compute on invalid research branches.

4.5 Separation of Concerns: Lead-PI vs Coder Subagents

The Iron Law: The Lead Principal Investigator (Lead-PI) NEVER WRITES implementation code.

• Lead-PI: Conducts parallel literature research via parallel-web, formulates hypotheses, issues structured task contracts, audits diffs/artifacts, and logs evidence verdicts.
• Coder Subagents: Execute strictly defined task contracts:

  ### Subagent Task Contract: [H-TAG]
  - IN Scope: [Target file, class, or function]
  - OUT of Scope: [Forbidden files or boundaries]
  - Goal / Metric Target: [Quantitative threshold, e.g., delta < -0.005 RMSLE]
  - Definition of Done: [Passing test suite, artifacts written to artifacts/]
  - Output Constraint: "Write digest to artifacts/<name>.md and return <= 10-line summary"
  

• Zero-Trust Summary Rule: The Lead-PI never trusts subagent summaries at face value. Code diffs, log outputs, and hash manifests must be inspected directly.

4.6 Zero-Overhead Epistemic Tracing

The AutoTracer module eliminates manual logging overhead. Every tool call and state transition is captured synchronously:

1. In the SQLite traces table;
2. In the GitHub-flavored Markdown table at docs/agent-trace.md with UTC timestamps, agent roles, and Git commit hashes:

4.7 Antifragile Dual-Mode Onboarding

Epires adapts seamlessly to any repository structure:

• Mode A: Clean / Empty Repositories: Interactive user interview ➔ directory scaffolding ➔ .epires/config.json initialization ➔ ready for baseline hypothesis $H_0$.
• Mode B: Existing / Custom Repositories: Topology reconnaissance ➔ dynamic path & metric inference ➔ interactive user alignment ➔ path binding in .epires/config.json and AGENTS.md ➔ historical hypothesis ingestion.

5. What's New in 0.4.0: Verifiable Epistemics

v0.4.0 turns the declared methodology into enforced, measurable machinery.

Computed, not declared

• Evidence Gates G0–G8 — evidence levels are the ceiling of passed predicates (provenance resolves, ≥3 seeds, held-out hash predates result, preregistration, CI95 outside significance threshold, …), recomputed automatically. EPIRES_STRICT_GATES=1 hard-fails violations.
• Duhem–Quine attribution — an anomaly refutes the bundle ⟨hypothesis + auxiliaries⟩, never the hypothesis alone. Single anomalies with suspects BLOCK without cascading; only reproduction on ≥2 independent axes (env/data/model/agent) falsifies and cascades.
• JTMS-lite + argumentation semantics — hypothesis status is computed from justifications and Dung grounded labeling (IN/OUT/UNDEC), not hand-set by agents.

Oversight (VSM)

• S3* independent auditor — s3_audit_confirmed() re-verifies every CONFIRMED hypothesis with a deterministic pre-test plus another model (EPIRES_AUDIT_MODEL).
• Algedonic channel — contradiction / audit-failure / repeated-failure triggers escalate past Lead-PI; freeze_branch() quarantines the affected DAG subtree.
• Calibration ledger — per-agent Brier/Platt tracking; calibrated_p(agent_id, stated_p) weights every claim by the agent's track record (<30 resolutions → skeptical prior).

Decision quality

• Experiment scoring by expected information gain — score_experiments(candidates, q) replaces intuition-driven selection.
• Pask conversations — CONFLICTS_WITH opens a structured conversation node (merge/split/add_condition) instead of a dead-end label.
• 28 MCP tools expose all of the above to coding agents.

6. CLI Command Reference

# Initialize Epires in the current repository
epires init [--dir <path>] [--force]

# Diagnostic doctor: verifies MCP tools, SQLite integrity, and agent configuration
epires doctor

# Output canonical JSON Schema and Python SDK migration snippet
epires schema [--format json|python]

# Bulk ingest historical hypotheses from Markdown, JSON, or JSONL
epires ingest [findings.md] [--dry-run] [--upsert]

# Generate a custom migration script template for proprietary notes formats
epires ingest --template scripts/migrate_findings.py

# Export research graph to portable JSON bundle with SHA256 checksum (for Git/CI)
epires export [--out research-graph.json] [--format json|jsonl]

# Import research graph from portable JSON bundle
epires import research-graph.json [--upsert]

# Perform reconnaissance scan on repository topology and domain
epires recon [--dir <path>]

# Start the MCP stdio server for AI agents
epires mcp

# Start the FastAPI REST API server & Web Dashboard
epires serve [--host 127.0.0.1] [--port 8000]

# Display hypothesis status in terminal
epires status

# Output Mermaid DAG diagram
epires dag

7. Model Context Protocol (MCP) Specification

Epires exposes 28 deterministic MCP tools for AI agents:

8. Eval Sandbox

sandbox/ measures whether prompts and models actually behave epistemically — scenarios with ground truth, graded 0–1 scoring, full transcripts.

# mock agents (no LLM needed)
python -m sandbox.run_eval --all --report

# real agents via opencode CLI
EPIRES_EVAL_MODEL=opencode/x-preview-f-free \
  python -m sandbox.run_eval --all --variant protocol --agent opencode

17 scenarios cover: Duhem–Quine attribution (planted_bug, hidden_confound, baseline_stale), selection bias (selection_bias, survivor_bias), metric traps (goal_metric_mismatch, seed_luck), leakage (leakage_gap, open_leak_hunt), test manipulation (planted_manipulation), premature commitment (commitment_trap, repro_flip), self-evaluation bias (double_blind_missing), vacuous confirmation (vacuous_confirm), drift (metric_drift), conflicts (conflicting, open_web_prior).

Measured result (protocol.md decision prompt): frontier-class model 15/15 PASS; free-tier models pass the easy half only — the battery discriminates.

9. Testing & Mathematical Fuzzing

The engine is verified using property-based fuzz testing powered by hypothesis:

pytest -v
============================== 38 passed in 4.08s ==============================

Verified Mathematical Invariants:

• VSA Invertibility: $\text{bind}(\text{bind}(\mathbf{a}, \mathbf{b}), \mathbf{b}) \equiv \mathbf{a}$ across arbitrary hypervectors.
• DAG Cascading Invariant: Falsifying any node in a random DAG strictly blocks all and only its transitive descendants (verified against BFS ground truth).
• Retraction & Unblock Invariant: Retracting a falsifying claim strictly unblocks all descendants whose remaining parents are valid.
• Storage Resilience: Complete tolerance to arbitrary Unicode payloads, extreme floats ($\pm 10^8$), and empty tags.

10. License

Distributed under the MIT License. See LICENSE for details.