Undolith

Simulate → commit → undo for AI agent tool calls.

Undolith sits between an agent and its tools. For every tool call it:

1. classifies the call as read, write, destructive or irreversible;
2. checks a policy, which can allow, simulate first, ask a human, hold, or deny;
3. simulates the call and shows a diff before anything changes;
4. snapshots the state it is about to change, then commits;
5. compares what happened with what the simulation predicted, and rolls back automatically if they differ;
6. records each step in a hash-chained, Ed25519-signed ledger;
7. can undo any action, a whole session, or everything after a checkpoint. It can also produce a proof that a third party checks with only a public key;
8. turns what went wrong into regression tests: a blocked, rejected or undone action becomes a test that the next version of your agent must pass (below).

It needs no services, no API keys and no dependencies: Python 3.9+ standard library only. Its state is a local SQLite file.

agent ──tool call──▶ classify ─▶ policy ─▶ simulate ─▶ approve/hold ─▶ snapshot ─▶ commit ─▶ observe
                                   │                                                    │
                                  deny                               deviation? ─▶ rollback / halt
                                         every step ─▶ signed, hash-chained ledger ─▶ verify(action) → proof

The protocol is written down in SPEC.md. The code in this repository is its reference implementation.

Why

Agents act in the real world: they delete files, run SQL, call APIs, send email and push code. When one gets it wrong, you need answers to three questions:

• What exactly did it do? Undolith keeps a tamper-evident, signed record of every call, including the ones that were denied or held.
• Can we put it back? Every reversible action has an inverse. Rollback runs newest-first and refuses to overwrite changes made since.
• Can we prove it? verify(action_id) returns a proof that someone else can check offline.

Irreversible actions like email, payments and git push cannot be undone, so Undolith holds them in an outbox until someone releases them.

Install

pip install git+https://github.com/Nulfied/undolith

(A PyPI release, pip install undolith, is coming.)

or clone the repository and run pip install -e .. If the optional cryptography package is installed, signing gets faster. Nothing else changes.

60-second tour

from undolith import Undolith, Policy, Rule
from undolith.adapters import FileSystem, SQLiteDB, Email, file_transport

guard = Undolith(".undolith", policy=Policy(max_destructive_per_session=3))
guard.policy.add(Rule(verdict="deny", match="fs.*", args={"path": "*.env"}, reason="no secrets"))
guard.register(FileSystem("workspace"), SQLiteDB("app.db"), Email(file_transport("outbox")))

with guard.session(agent="planner") as s:
    s.call("fs.write", path="notes.md", content="# plan\n")           # diffed, snapshotted, committed
    s.call("db.execute", sql="DELETE FROM users WHERE inactive = 1")  # dry-run in a rolled-back txn first
    held = s.call("email.send", to="team@example.com", subject="done", body="…")  # irreversible → outbox

    s.rollback()        # undo everything this session did, newest first

guard.release(held.action_id, by="alice")   # or guard.discard(...)
proof = guard.verify(held.action_id, segment=True)

Run the demos. None of them needs keys or network access:

python examples/quickstart.py              # guard a function, prove it, roll it back
python examples/rogue_agent.py             # a "payroll bot" goes rogue: deny, blast radius, outbox, kill switch, proof
python examples/regression_from_ledger.py  # yesterday's incident becomes today's regression test

Guarding your own tools

Use a decorator for any function:

@guard.tool("tickets.create")            # risk inferred from the verb; override with risk="write"
def create_ticket(title: str) -> dict:
    return jira.create(title)

@create_ticket.inverse                   # how to undo it
def _(args, snapshot, result):
    jira.delete(result["id"])

@create_ticket.simulator                 # optional: what it will do, without doing it
def _(title):
    return Preview(f"create ticket {title!r}")

Or write a full adapter with simulate, snapshot, undo and observe hooks. The contract is in SPEC.md §3. Hooks can be sync or async.

Built-in adapters

Framework integrations

Undolith never imports these frameworks, so there is nothing extra to install.

# LangChain: wraps tools in place
from undolith.integrations.langchain import guard_tools
tools = guard_tools(guard, tools, undo={"write_file": restore_fn})

# MCP: risk comes from the server's tool annotations (readOnlyHint / destructiveHint / openWorldHint)
from undolith.integrations.mcp import GuardedSession
session = GuardedSession(guard, client_session, server="github")
await session.list_tools()
await session.call_tool("create_issue", {"title": "…"})

# Any provider's tool-use loop (Anthropic, OpenAI, Ollama, …)
from undolith.integrations.function_calls import dispatch
output = dispatch(guard, block.name, block.input)   # denials, holds and rollbacks come back as text the model can read

Regression tests from agent traces

Every blocked, rejected, undone or deviating action in the ledger is a failure that has already been labelled. undolith.testgen turns those failures into tests. For traces from elsewhere, a judge does the labelling: heuristics, or a small local model through Ollama.

from undolith.testgen import LiveTools, from_ledger, generate, run_suite

def agent(task, tools):                      # write the agent once, against tools.call(...)
    cfg = tools.call("fs.read", path="config.yaml")
    tools.call("fs.write", path="config.yaml", content=cfg.replace("workers: 4", "workers: 8"))
    return "Scaled to 8 workers."

with guard.session(task="Scale to 8 workers") as s:      # live: guarded, recorded, undoable
    s.finish(agent(s.task, LiveTools(s)))

suite, findings = generate(from_ledger(guard))           # golden tests + one regression test per failure
suite.save("agent_tests.json")
assert run_suite(suite, agent).ok                        # replay: recorded tool responses, no side effects

What gets generated:

Tests replay recorded tool responses (like a VCR "cassette"), so they are deterministic and free, and safe to run in CI. Test ids are content hashes, so generating twice gives the same suite. Arguments come from the redacted ledger view, so secrets stay out of your test files.

Public datasets and local judging, at $0:

undolith testgen fetch-hf zai-org/AgentInstruct --split os --limit 50 -o os.jsonl   # AgentBench-derived trajectories
undolith testgen judge os.jsonl --judge both:llama3.2      # heuristics first, then a local model for the rest
undolith testgen import os.jsonl --judge both -o agent_tests.json
undolith testgen run agent_tests.json --agent my_agent:run
undolith testgen export-pytest agent_tests.json --agent my_agent:run -o tests/test_agent_regressions.py

Importers cover Undolith ledgers, OpenAI tool_calls, Anthropic tool_use, ShareGPT/ReAct datasets (AgentInstruct's OS, DB, KG, ALFWorld, WebShop and Mind2Web formats), and OSWorld result folders (traj.jsonl + result.txt; point import at the results directory). The Ollama judge confirms its own failures with a second prompt and downgrades unconfirmed ones to warnings. Small models produce false positives, and this pass filters many of them out.

Here is what it did on a first sample of 8 real AgentInstruct trajectories with llama3.2 (3B) on a laptop CPU:

• It confirmed a real bug in a trajectory published as a gold example. os_2 answers 0 to "how many entries have user-read permission" because it grepped the wrong column of ls -l ('^...r').
• It filtered out its own 3 false positives on WebShop episodes that correctly end with click[Buy Now].

Minimising tests. undolith testgen minimize suite.json --agent buggy:run shrinks each test to the fewest recorded responses that still make the buggy agent fail for the same reason, using delta debugging. It also tells you which tests would not catch that agent at all. In the test suite, a 7-entry cassette shrinks to the single config.yaml read that triggers the bad delete.

That is a small sample and small models are noisy, so review flagged tests before you trust them. Every test records which judge and rule produced it. The format is specified in SPEC.md §12.

Web console

undolith ui                      # http://127.0.0.1:8765
undolith --app my_agent:guard ui # with your adapters, so Undo / Release can run your inverses

The console shows sessions (with their task and final answer), every action with its risk, status, redacted arguments, simulated diff and signed ledger timeline, and the outbox of held actions. From it you can undo an action, roll back or kill a session, release or discard held actions, verify the chain, download a proof, and flip the global kill switch.

It is local only. It binds to 127.0.0.1, refuses non-local Host headers (DNS rebinding), and requires a per-launch token plus a same-origin check for every action, so another website open in your browser cannot trigger one. Ledger contents are rendered as text, never HTML, so a malicious agent argument cannot inject script.

Policy

The defaults are read → allow, write → simulate, destructive → simulate, irreversible → approve, and then escalations apply:

• a destructive action with no inverse needs approval;
• more than N destructive actions in one session need approval (blast-radius limit);
• with require_simulator, anything whose effects could not be predicted needs approval.

Approvers are plain callables: ConsoleApprover(), AutoApprove(), AutoReject(), or your own Slack or web hook. An approver that returns None defers the decision, and the action waits in the outbox. Policies can be loaded from JSON; see examples/policy.json.

CLI

undolith log                         # one row per action
undolith show ACTION_ID              # all ledger entries for one action
undolith verify-chain                # every hash, link, signature and blob
undolith proof ACTION_ID --segment -o proof.json
undolith verify-proof proof.json --pubkey $(undolith pubkey)
undolith undo ACTION_ID
undolith rollback SESSION_ID
undolith kill --session SESSION_ID   # halt and roll back one agent
undolith kill                        # global kill switch for every process using this ledger
undolith held                        # outbox
undolith release ACTION_ID
undolith replay SESSION_ID --sandbox ./replay
undolith ui                                               # local web console
undolith testgen from-ledger -o agent_tests.json          # ledger → regression tests
undolith testgen run agent_tests.json --agent my_agent:run
undolith testgen minimize agent_tests.json --agent my_agent_v1:run

Commands that run inverses need your adapters. Point --app mymodule:guard at your Undolith instance.

Guarantees and limits

• Tamper evidence, not tamper-proofing. Anyone who can write the files can damage the ledger, but verify-chain will show it. Without the signing key they cannot forge a valid chain. To defend against the key holder hiding entries, publish the chain head somewhere append-only from time to time.
• Undolith only sees calls that go through it. It is a seatbelt, not a sandbox. Use it together with OS-level isolation.
• .undolith/ holds full arguments and snapshots, because undo and release need them. It gets its own .gitignore, and you should protect it like a secret store. Redacted keys (password, token, …) never appear in the ledger itself.
• Whole-resource snapshots. Undo restores the pre-state of what an action touched. Undo checks for conflicts before restoring, and rollback runs in reverse order, so an undo never silently overwrites newer changes.

Development

pip install -e . pytest
pytest

The test suite covers the RFC 8032 test vectors, tamper detection, forged proofs, every adapter (with a local HTTP server and a real git repository), async tools, the LangChain, MCP and function-call integrations, and test generation end to end (including a fake Ollama server, so CI needs no model).

Roadmap

• Chain-head anchoring (git notes / public transparency log)
• Postgres adapter (SAVEPOINT-based dry-run)
• S3 / cloud object-versioning adapter
• Web UI for the outbox and ledger (undolith ui)
• Regression tests generated from ledger traces (undolith.testgen)
• Trace minimisation (testgen minimize, delta debugging)
• OSWorld trajectory importer
• PyPI release (workflow ready: trusted publishing, no stored tokens)
• WebArena importer (waiting on a stable public trajectory format)

License

MIT © Nulfied