Sembl

A deterministic accountability gate for AI coding agents.

Sembl checks what an AI coding agent actually changed against the bounds the change was supposed to stay in — which files it may touch, which it must not, how big the diff may get — and whether it lied about what it touched or what it tested. It runs after the agent and before a human approves, and returns a single verdict: PASS / WARN / BLOCK.

declared bounds  +  the agent's real diff  ->  sembl verify  ->  PASS / WARN / BLOCK

The check is deterministic (no LLM in the loop), executor-neutral (Cursor, Claude Code, Codex, Aider, OpenCode — Sembl never sees the agent), and free to run in CI. Same inputs, same verdict, every time.

Website | PyPI | Issues

Why

AI agents are strong executors. The gap is no longer writing the code — it is trusting what the agent did without reading every line. As agents get more autonomous and humans review less, you need a repeatable, mechanical check that the change stayed inside its declared bounds and didn't fabricate its results.

What Sembl does: deterministically catch objective, checkable problems in a diff — out-of-scope edits, edits in forbidden areas, files the agent claimed to change but didn't, and "tests passed" claims with no evidence.

What Sembl does not do: it does not make the agent smarter, and it does not judge code quality or maintainability. That's a reviewer's job, and an LLM reviewer does it about as well — so it isn't Sembl's claim. Sembl's edge is narrow and honest: determinism, zero cost, and auditability — a gate you can put in CI that gives the same answer every time and that nobody can argue with.

How it works

1. Declare the bounds of the change — the files it may edit, areas it must not touch, and (optionally) a size budget. Produce this with whatever you already use (see Declaring bounds).
2. Your agent does the work.
3. sembl verify compares the real git diff against the bounds and returns PASS / WARN / BLOCK. With --report, it also cross-checks the agent's own success report and flags fabricated file claims and unevidenced test passes.

pip install sembl

# after your agent has edited the repo:
sembl verify --wo-file bounds.json --report agent-report.json

sembl verify — BLOCK
  Files changed         7
  Scope (out of scope)  infra/deploy.yaml
  Forbidden hits        none
  Fabricated claims     src/payments/refund.ts
  Validation evidenced  missing: pytest
  Churn vs budget       7 files > 6
  Reasons               • fabricated claims (reported but unchanged): src/payments/refund.ts
                        • out-of-scope edits: infra/deploy.yaml

Exit codes: PASS=0, WARN=0 (or 1 with --strict), BLOCK=1. Add --json for machine-readable output in CI.

Declaring bounds (bring your own)

verify reads a small, portable contract. Only four fields matter — anything that can emit them can drive the gate:

{
  "editable_paths": ["src/auth/"],
  "forbidden_areas": ["migrations/", "infra/"],
  "churn_budget": { "max_files": 6, "max_lines": 200 }
}

Produce it however you like:

• GitHub Spec Kit (recommended). If you plan changes with Spec Kit, its tasks.md already lists the exact file paths for each task — those map straight onto editable_paths. Use Spec Kit (or Tessl, or Kiro) to decide what to build; use Sembl to verify the agent stayed in those lines. They are complementary: spec tools live upstream of the agent, Sembl lives downstream of it.
• By hand. Write the four fields yourself. It's a normal JSON file.
• sembl generate (beta, optional). Sembl can also produce a fuller Work Order JSON that verify reads directly — see Beta: Work Order generation. This is optional and interchangeable with the tools above.

What verify checks (all deterministic)

Tests are always allowed alongside in-scope edits. The verdict rolls up to BLOCK (hard contract breach), WARN (soft signal), or PASS (clean). --strict makes WARN a failing exit code for CI gating.

The executor report (the --report file) is JSON the agent or your harness emits; Sembl never trusts it — it only uses it to catch contradictions with the real diff. Common shapes are recognized (files_modified, changes[], tests_passed, a checks[] list, …).

Gate it in CI

verify can score a PR's diff directly — no working-tree checkout needed — so it drops into CI as a hard gate. Feed it a patch with --diff (or --diff - for stdin):

git diff origin/main...HEAD | sembl verify --wo-file bounds.json --diff - --strict

Or use the GitHub Action on every pull request (see examples/github-workflow.yml):

- uses: actions/checkout@v4
  with: { fetch-depth: 0 }
- uses: speedvibecode/sembl@v0.1.14
  with:
    bounds: bounds.json
    strict: "false"   # advisory by default; true for a hard scope gate

Install

pip install sembl

# As an isolated tool
uv tool install sembl

From source:

git clone https://github.com/speedvibecode/sembl
cd sembl
uv pip install -e .

Status & honesty

Sembl went through a long generation-first phase. We tested its central premise — that compiling a rich up-front contract makes the agent produce better outcomes — and our own results did not support it: a one-line task did as well as an eight-part Work Order for a capable agent. So we stopped claiming it.

What survived testing, and what Sembl now is:

• verify — the accountability gate. This is the product. Deterministic checks over a real diff; covered by the test suite.
• generate / clarify / graph context — beta, optional, unproven as an outcome-improver. Kept because they're a convenient way to produce a bounds file, but if you already use GitHub Spec Kit / Tessl / Kiro, use that instead and feed its output to verify.

We make no claim that Sembl improves what an agent produces. Its claim is narrow and true: a repeatable, executor-neutral, free check that a change stayed in declared bounds and didn't fabricate its results.

Beta: Work Order generation

Optional and unproven. Everything in this section is the older generation half of Sembl. It produces a "Work Order" — a scoped contract + executor prompt

• a work-order.json that verify can read directly. It is interchangeable with GitHub Spec Kit and similar tools; reach for it only if you don't already have a spec/planning step.

# core (generation uses an LLM provider)
pip install sembl

# with optional graph context (Graphify + code-review-graph)
pip install "sembl[graph-pipeline]"

Set one provider key, then generate:

$env:OPENAI_API_KEY="..."   # or ANTHROPIC_API_KEY / GEMINI_API_KEY / NVIDIA_API_KEY / OPENROUTER_API_KEY
sembl generate --repo C:\path\to\repo --task "fix the failing login redirect test" --provider openai

Providers: openai, anthropic, gemini, nvidia, openrouter, tokenrouter, local ollama, and claude-cli. Output lands in .sembl/work-orders/wo-...{slug}/ as work-order.md, executor-prompt.md, validation-plan.md, and work-order.json (the file verify --wo-file reads).

Other beta commands:

sembl clarify  --repo . --task "..."   # judge whether a task is specified enough to scope (exit 2 if blocked)
sembl doctor   --repo .                # check the optional graph subsystem (tools, graphs, keys, fixes)
sembl show                             # show the latest Work Order
sembl list                             # list Work Orders in this repo
sembl validate                         # older PASS/FAIL form of verify (kept for back-compat)

Optional graph context

generate can use Graphify and code-review-graph to ground scope in the repo's structure. It is best-effort and off the critical path; run sembl doctor to see what's installed and how to fix gaps, and use --graph-mode auto|required|off to control it.

Local test

.venv\Scripts\python.exe -m pytest tests\ -q
python -m compileall -q sembl tests

Releasing

Publishing uses GitHub Actions and Trusted Publishing (OIDC); no API tokens are stored. .github/workflows/release.yml publishes to PyPI when you publish a GitHub Release whose tag (v<version>) matches pyproject.toml and sembl/__init__.py. See VERSIONING.md.

Models write code. Sembl makes the change accountable.