scitex-clew 0.2.5

Verifiable knowledge graph for scientific experiments

SciTeX Clew (scitex-clew)

Verifiable knowledge graph for scientific experiments

Full Documentation · pip install scitex-clew

---

Problem

Scientific publications are growing exponentially — accelerated by LLM-assisted writing — yet peer review remains a manual bottleneck. 70% of researchers report failed replication attempts, and only 11-36% of high-profile findings are successfully reproduced. Existing tools (pre-registration, containerization, workflow managers) address whether research could be reproduced, but not whether it has been.

Solution

SciTeX Clew records every artifact produced during research — code, data, figures, statistics — into a hash-linked DAG (directed acyclic graph). This creates a verifiable knowledge graph of scientific experiments, which can be explored by humans or AI agents.

Named after the thread Ariadne gave Theseus to trace his path through the labyrinth, Clew serves two purposes:

1. Reproducibility verification — confirm that outputs remain unchanged and that every step in the pipeline is intact.
2. Research logic comprehension — visualize and navigate the structural skeleton of a research project, from raw data through analysis to manuscript claims.

The DAG is a structured, machine-readable representation of an entire research project — enabling both human reviewers and AI agents to inspect, verify, and understand the logic programmatically. It lets you:

• Verify that outputs remain consistent with recorded hashes
• Trace provenance chains from any file back to its source
• Visualize the structural logic of a research project as a navigable graph
• Re-execute scripts in a sandbox to confirm reproducibility
• Link manuscript claims to the computational sessions that produced them

Five Node Classes

Every node in the DAG is classified into one of five semantic roles:

Class	Role	Examples
Source	Data acquisition scripts	01_download.py, collect.sh
Input	Raw data and configuration	raw_data.csv, config.yaml
Processing	Transform and analysis scripts	03_analyze.py, train.R
Output	Intermediate and final data products	results.csv, figure1.png
Claim	Manuscript assertions tied to evidence	"Fig 1 shows p<0.05", "Table 2"

_{Table 1. Five node classes. Classification is inferred automatically from file extensions and session roles, or set explicitly via set_node_class().}

This classification turns the DAG into a navigable map of the research project. The key operation is backpropagation from claims to sources: starting from a manuscript assertion (claim), Clew traces backward through outputs, processing scripts, and inputs to the original raw data — verifying every hash along the way.

Three Verification Modes

Mode	Scope	API	Description
Project	Entire pipeline	clew.dag()	Verifies every session recorded in the database in topological order. A navigation map for ongoing project monitoring. Answers: "Is the whole project intact?"
Files	Specific outputs	clew.dag(["output.csv"])	Traces backward from target files through their dependency chain and verifies each session. Answers: "Can I trust this specific file?"
Claims	Manuscript assertions	clew.verify_claim("Fig 1")	Verifies individual claims linked to source sessions. Answers: "Is this figure/statistic still backed by the data?"

_{Table 2. Three verification modes. Each mode supports both cache verification (millisecond hash comparison) and re-run verification (sandbox re-execution with rerun_dag / rerun_claims).}

Installation

Requires Python >= 3.10. Zero dependencies — pure stdlib + sqlite3.

pip install scitex-clew

SciTeX users: pip install scitex already includes Clew. Tracking is automatic via @scitex.session + scitex.io.

Quickstart

import scitex_clew as clew

# Git-status-like overview
clew.status()

# Verify a run (hash check)
result = clew.run("session_20250301_143022")

# Trace a file's provenance chain
chain = clew.chain("output/figure.png")

# Verify the full DAG
dag_result = clew.dag(["output/figure.png"])

# Re-execute in sandbox and compare
rerun_result = clew.rerun("session_20250301_143022")

_{Figure 1. Example DAG visualization. Green nodes indicate verified sessions; red nodes indicate hash mismatches. Clew traces the dependency graph backward from target files to raw data sources.}

Four Interfaces

Python API

import scitex_clew as clew

clew.status()                              # overview
clew.run("session_id")                     # verify one run
clew.chain("output/figure.png")            # trace provenance
clew.dag(["output/figure.png"])            # verify full DAG
clew.rerun("session_id")                   # sandbox re-execution
clew.mermaid(claims=True)                  # Mermaid DAG diagram
clew.add_claim("Fig 1 shows p<0.05", source_files=["fig1.png"])

Full API reference

CLI Commands

clew --help-recursive                      # Show all commands
clew status                                # Git-status-like overview
clew verify <session_id>                   # Verify a run
clew list                                  # List tracked runs
clew stats                                 # Database statistics
clew mermaid                               # Generate Mermaid diagram
clew list-python-apis                      # List Python API tree
clew mcp list-tools                        # List MCP tools

Full CLI reference

MCP Server — for AI Agents

AI agents can verify reproducibility and trace provenance autonomously.

Tool	Description
clew_status	Git-status-like overview
clew_run	Verify a specific run
clew_chain	Trace file provenance chain
clew_dag	Verify full DAG
clew_list	List tracked runs
clew_stats	Database statistics
clew_mermaid	Generate Mermaid DAG diagram
clew_rerun_dag	Rerun full DAG in sandbox
clew_rerun_claims	Rerun all claim-backing sessions

_{Table 3. Nine MCP tools available for AI-assisted verification. All tools accept JSON parameters and return JSON results.}

clew mcp start

Full MCP specification

Skills — for AI Agent Discovery

Skills provide workflow-oriented guides that AI agents query to discover capabilities and usage patterns.

clew skills list              # List available skill pages
clew skills get SKILL         # Show main skill page
scitex-dev skills export --package scitex-clew  # Export to Claude Code

Skill	Content
quick-start	Basic API, session tracking, first verification
cli-commands	CLI reference (clew status, clew verify, etc.)
mcp-tools-for-ai-agents	MCP tool reference for AI agents
common-workflows	Claims, DAG patterns, stamps, reproducibility

Part of SciTeX

Clew is part of SciTeX. When used inside the SciTeX framework, tracking is automatic:

import scitex

@scitex.session
def main(CONFIG=scitex.INJECTED):
    data = scitex.io.load("input.csv")    # auto-tracked as input
    result = process(data)
    scitex.io.save(result, "output.csv")   # auto-tracked as output
    return 0

All file I/O through scitex.io is recorded in the clew database:

scitex.clew.status()              # overview
scitex.clew.run("session_id")     # verify
scitex.clew.mermaid(claims=True)  # DAG diagram

The SciTeX system follows the Four Freedoms for Research below, inspired by the Free Software Definition:

Four Freedoms for Research

0. The freedom to run your research anywhere — your machine, your terms.
1. The freedom to study how every step works — from raw data to final manuscript.
2. The freedom to redistribute your workflows, not just your papers.
3. The freedom to modify any module and share improvements with the community.

AGPL-3.0 — because we believe research infrastructure deserves the same freedoms as the software it runs on.

---