Skip to main content

Tenthcheck

Tenthcheck, from Tenthlook: see what an AI coding agent's edit could break before it lands.

Static checks for Python repositories, as a CLI and an MCP server for coding agents.

Held-out accuracy: 53 of 60 true direct callers found (88%), 0 false positives, on starlette (271c277), a library no fix was made for. The 23 symbols were picked by a fixed call-frequency rule, every call site was labelled by hand from grep hits without seeing Tenthcheck's output, and the labels and the parser were hashed before scoring once (measurements/results/accuracy_starlette/). Small sample, typed code only; details and the other two libraries are under Measured.

Known misses on that set (no edge is produced; nothing is guessed):

  • loops over self.routes or [] where routes is a property, or over an attribute set by a if c else b (3)
  • calls on a value returned by a @property, e.g. self.headers.getlist() (2)
  • a union annotation, Router | Starlette | None: only Router's method got the edge (1)
  • an unannotated attribute, self.router = Router(...), used in another method (1)

Still missed on click: a call inside if isinstance(command, Group): and one through a closure variable. The full list of what isn't resolved is in What this does not do.

  • tenthcheck check <repo>: import cycles among the repo's modules plus a summary (modules, functions, classes and edges parsed; files skipped for syntax errors).
  • tenthcheck check <repo> <file> <proposed_file>: parse the proposed version of one file in memory and report new import cycles plus the dependents of every function or class it changes. Exit code 1 if a new cycle appears, 2 on a syntax error in the proposed file.
  • tenthcheck blast <repo> <symbol>: the functions, classes and modules that call, subclass or import <symbol>, grouped by hop.
  • tenthcheck cycles <repo>: import cycles among the repo's modules, each printed as a path.
  • tenthcheck scope <before_dir> <after_dir>: changed functions/classes between two snapshots and the untouched code that depends on them.
  • tenthcheck serve [repo]: an MCP stdio server named tenthcheck for repo (default: the current directory), with four tools: check_blast_radius, find_import_cycles, check_proposed_edit, scope_audit.

Scoring is plain graph search over Python's ast; no model is called. It does not run your code.

Install

pip install tenthcheck          # CLI, no dependencies (Python >= 3.10)
pip install "tenthcheck[mcp]"   # adds the official MCP Python SDK for `tenthcheck serve`

Set up in Claude Code

Install with the [mcp] extra so the tenthcheck command is on your PATH, then either:

  • add it from the repo you want checked (project scope writes .mcp.json there):

    claude mcp add --scope project tenthcheck -- tenthcheck serve
    
  • or put this in .mcp.json at the repo root:

    {"mcpServers": {"tenthcheck": {"type": "stdio", "command": "tenthcheck", "args": ["serve"]}}}
    

tenthcheck serve checks the directory it is started in; Claude Code starts project-scoped servers in the project root. To check a different directory, pass it: "args": ["serve", "path/to/src"]. Run /mcp in Claude Code to confirm the tenthcheck server is connected. To have the agent use it, ask it to call check_proposed_edit before writing each file.

Example

A constructed edit to httpx (b5addb6) that adds from ._urls import URL to httpx/_urlparse.py:

$ tenthcheck check . httpx/_urlparse.py proposed_urlparse.py
File: httpx/_urlparse.py
NEW IMPORT CYCLES: 1
  httpx._urlparse -> httpx._urls -> httpx._urlparse   [added by this edit: httpx._urlparse -> httpx._urls]
Changed symbols: 0 (modified 0, added 0, removed 0)
$ echo $?
1

Measured

All numbers below come from files in measurements/results/, produced by the scripts in measurements/scripts/. Those files were recorded before the rename and use the package's previous name (see CHANGELOG.md); the numbers are unchanged. Libraries were shallow clones at the SHAs in measurements/results/corpus.txt (click and starlette: their truth.json), parsed at the repo root (tests included). Machine and Python version are in measurements/results/latency.json.

Parsing real libraries (measurements/scripts/harden.py -> measurements/results/hardening.json)

Repo (whole clone) .py files Parse time Functions Classes CALLS edges IMPORTS edges Import-cycle groups Crashes / skipped files
httpx 60 0.47 s 1133 107 370 319 0 0 / 0
flask 83 0.46 s 1457 154 231 424 1 0 / 0
pydantic 3156 7.36 s 8241 9907 1731 13187 0 0 / 0

Re-run 2026-11-12 with the typed-call pass (below); CALLS edges rose from 259 / 171 / 1,472 (round 1) to the numbers shown, and parse time rose about 0.1 s on httpx and flask and ~1 s on pydantic (parse times in this table: measured on battery; may be slower on AC). The typed pass raised no errors on any of the four libraries parsed. pydantic's clone is mostly tests/ (2,961 of 3,156 files). Dependents walks never hit the 10,000-node cap (hit_node_cap = 0 in every repo).

Latency (measurements/scripts/bench_latency.py -> measurements/results/latency_2026-11-17_ac.json)

Measured 2026-11-17 on the current code, on AC power for the whole run (measurements/scripts/ac_bench.sh checked the power source every 5 s; also recorded in the file's meta.power). Apple M4, Python 3.11.18. Milliseconds. MCP (warm): one serve process per repo, real stdio round trips through the official MCP client, after one untimed call per tool. CLI (cold): a fresh process each time, parse included. With n = 5–100 runs, p99 is close to the max; n is shown.

Repo Tool MCP p50 MCP p99 n CLI cold p50 CLI cold p99 n
httpx check_blast_radius / blast 3.7 5.2 100 601 711 5
httpx find_import_cycles / cycles 2.6 3.8 30 630 646 5
httpx check_proposed_edit / check 321.1 420.8 20 1216 1268 5
httpx scope_audit / scope 1434.6 1531.3 5 1274 1351 5
flask check_blast_radius / blast 6.8 8.9 100 596 610 5
flask find_import_cycles / cycles 5.8 7.3 30 534 559 5
flask check_proposed_edit / check 339.4 427.6 20 994 1074 5
flask scope_audit / scope 1480.5 1760.0 5 992 1031 5
pydantic check_blast_radius / blast 119.9 813.2 100 8605 8880 5
pydantic find_import_cycles / cycles 100.9 716.9 30 8712 9226 5
pydantic check_proposed_edit / check 4527.6 10625.0 20 17900 18276 5
pydantic scope_audit / scope 30128.2 32017.7 5 20002 20371 5

The first MCP call in a session parses the repo: httpx 60 ms, flask 11 ms, pydantic 97 ms. That first number is lower than the cold parse time above because the server already parsed the repo once at startup, before the client's first timed call. On pydantic, a warm blast call spends about 55 ms re-checking every file's mtime and about 35 ms rebuilding the reverse edge index for the walk (measurements/results/pydantic_warm_breakdown.txt, medians of 10, measured 2026-11-11). check_proposed_edit rebuilds the whole graph from cached syntax trees (~4.5 s on pydantic). scope_audit parses two full snapshots from disk on every call.

Run-to-run variance is large on this laptop. A battery run of the same code a few hours earlier (measurements/results/latency_2026-11-17.json) was faster than this AC run in 20 of 24 rows, by up to a third (flask warm blast: 4.5 vs 6.8 ms p50; pydantic warm scope_audit: 23.2 vs 30.1 s). The 2026-11-11 run, before the typed-call pass (measurements/results/latency.json, power source not recorded), was faster again: warm check_proposed_edit on httpx was 185 ms then, 297 ms on battery and 321 ms on AC today. Some of that is likely the typed-call pass, but with this much variance between same-day runs these files do not show how much. Treat the table as one run, not a precise benchmark.

How complete are the edges?

Calls: hand checks against true callers (measurements/scripts/accuracy_eval.py). For each symbol, every textual call site in the package was listed with grep (measurements/scripts/accuracy_candidates.py) and read to decide whether it really calls that symbol; the true direct callers were then compared with blast's hop 1.

Set Role Round 1 parser + typed-call pass + overload / super() / tuple fixes
httpx, 10 symbols, 28 callers (measurements/results/accuracy_httpx.json) tuned-on (its misses designed the typed pass) 21/28 (75%) 28/28 28/28
click, 14 symbols, 37 callers (measurements/results/accuracy_click/truth.json) held out: labelled and hashed before any change, measured once 12/37 (32%) 24/37 (65%) 35/37 (95%), now tuned-on
starlette, 23 symbols, 60 callers (measurements/results/accuracy_starlette/truth.json) held out: labelled and hashed 2026-11-17 against the current code, measured once not run not run 53/60 (88%)

False positives were 0 in every column except one on click before the overload fix. Files: measurements/results/accuracy_baseline_before_fixes.json, measurements/results/accuracy_after_typed_calls.json, measurements/results/accuracy_after_click_fixes.json; hashes and dates in measurements/results/accuracy_click/FROZEN.sha256. The starlette run (all three sets on the current code, 0 false positives) is measurements/results/accuracy_starlette/eval_2026-11-17.json; its hashes are in measurements/results/accuracy_starlette/FROZEN.sha256 and its symbol-selection rule is measurements/scripts/accuracy_select.py.

How to read this. For the current code the held-out number is 88% (53/60 on starlette, a library no fix was made for). Click's 95% came after fixes found by looking at click's misses, so it is not held out; click's held-out number (65%) was for the earlier typed-call pass. Starlette's symbols were picked by a fixed frequency rule and labelled from grep hits without seeing Tenthcheck's output, by Claude, which also wrote the fixes. All three checks are small (47 symbols, 125 callers in three libraries) and only cover code with type annotations.

What the 7 round-1 misses were (httpx): all were methods called on a typed object other than self: pattern.matches() on a loop variable over an annotated dict (2), response.read() on a value returned by an annotated method (3), from with client.stream() as response (1), and from response = yield request in an annotated generator (1). None were decorators, aliased imports or true dynamic dispatch. The typed-call pass (src/tenthcheck/typed_calls.py) fixes these by reading annotations.

What the 13 held-out misses were (click, after the typed pass): @typing.overload stubs took the method's name, so calls attached to the wrong definition (10); super().method() (1); a tuple-unpacked annotated return (1); isinstance narrowing (1). The first three are now fixed; isinstance narrowing is not.

What the 7 held-out misses were (starlette, current code): loops over self.routes or [] where routes is a property, or over an attribute set by a conditional expression (3); calls on a @property's return value, self.headers.getlist() (2); a union annotation, Router | Starlette | None, where only Router's method got the edge (1); an unannotated self.router = Router(...) attribute (1). None are fixed.

Imports: compared with a runtime trace (measurements/scripts/trace_imports.py, measurements/scripts/compare_imports.py -> measurements/results/imports_vs_runtime_*.json). Each package was imported module by module under an __import__ hook that records real module-level imports.

Package Modules Runtime edges Also found statically Static-only Cycle groups match
httpx 23 80 79 0 yes (none)
flask 24 72 72 0 yes (one group of 9 modules)

The httpx miss is httpx -> httpx._main, an import inside try: (see limits). pydantic was not traced: it needs its compiled core built, so its cycle result has no runtime check.

Tested in Claude Code

One headless Claude Code session (2026-11-12) did a real refactor on click with only this MCP server connected: it called check_proposed_edit before writing each file and check_blast_radius afterwards, and the answers were correct for that edit. It also exposed a bug (overload stubs named by line, so one inserted line made 24 untouched symbols look changed), fixed the same day. It cost $2.95 and ran 21 turns. Write-up and resulting diff: measurements/results/claude_code_mcp_session/ (the raw transcript is not published; it contains local paths). One session on a small, safe refactor; it shows the integration works, not that it changes agent behaviour.

What this does not do

  • No semantic checks. It does not know what your code means, does not run it, and does not tell you whether a change is correct. Run your tests.
  • Misses calls it can't type statically. Resolved: bare names, module.func, self.method() (including inherited), super().method(), and obj.method() when obj's class is known from a parameter or attribute annotation, a constructor, an annotated return value, with ... as, a loop over an annotated container, or yield in an annotated generator. Not resolved (no edge, never a guess):
    • untyped parameters and attributes (def f(x): x.read()), and values from untyped calls;
    • isinstance narrowing (if isinstance(cmd, Group): cmd.resolve_command());
    • closures that use a variable from the enclosing function;
    • getattr, operator.methodcaller, calls through dicts of functions, callbacks and plugin registries;
    • decorators that replace a function (the call graph sees the original definition);
    • generic type parameters bound at the call site (T in def first(xs: list[T]) -> T);
    • anything whose type comes from a package that wasn't parsed;
    • when a variable is reassigned to different types in one function, the last assignment wins;
    • values returned by a @property, union annotations (on starlette only one member got the edge), unannotated self.x = Cls(...) attributes used in other methods, and a or b / a if c else b expressions. On click, these left 2 of 37 callers unfound (one closure, one isinstance narrowing); on starlette, 7 of 60.
  • Only module-level imports. Imports inside functions, if, try or if TYPE_CHECKING: blocks are not edges (imports under if TYPE_CHECKING: are used to resolve type annotations only). This hides the usual deliberate cycle-breakers, and it also hides real imports inside try.
  • Star imports (from x import *) record the module edge but bind no names.
  • Python only. No C extensions, stubs, notebooks, or other languages.
  • Not a judgement on cycles. Many import cycles through a package __init__ are intentional and harmless (flask has one). check reports new cycles; whether one matters is your call.
  • Names are relative to the directory you point it at. Run it on the directory that contains your package (flask/src, not flask) to get ids like flask.app.
  • Ambiguous symbols are refused. blast with a name that matches several definitions lists them and exits 2; pass the full dotted id.

Development

pip install -e ".[mcp]" pytest
python -m pytest tests            # the MCP test is skipped without the [mcp] extra

The analysis core (graph, parser, typed_calls, topology, scope_audit, cycles, check_edit, index) is developed in Tenthlook's research codebase and copied here; the first line of each file names its source. Bug reports and fixes are welcome here.

Reproducing the measurements. The scripts in measurements/scripts/ expect clones under ~/tenthcheck-corpus/<name> at the SHAs in measurements/results/corpus.txt (click and starlette: their truth.json), e.g. git clone https://github.com/encode/httpx ~/tenthcheck-corpus/httpx && git -C ~/tenthcheck-corpus/httpx checkout b5addb6. Run them from measurements/:

cd measurements
python scripts/accuracy_eval.py ~/tenthcheck-corpus
python scripts/harden.py ~/tenthcheck-corpus results/hardening.json
python scripts/bench_latency.py ~/tenthcheck-corpus results/latency.json

On a macOS laptop, scripts/ac_bench.sh [corpus_dir] [python] runs the latency benchmark only while on AC power: it waits for 2 minutes of uninterrupted AC and discards the run if the power source changes.

To check the held-out freeze: cd measurements && grep results/ results/accuracy_starlette/FROZEN.sha256 | shasum -a 256 -c, and from the repo root grep src/ measurements/results/accuracy_starlette/FROZEN.sha256 | shasum -a 256 -c.

MIT licensed.

Metadata

Release files for tenthcheck 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tenthcheck 0.1.1
File Size Uploaded Provenance Actions
tenthcheck-0.1.1.tar.gz 43.9 kB Without provenance Details for tenthcheck-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for tenthcheck 0.1.1
File Interpreter ABI Platform Actions
tenthcheck-0.1.1-py3-none-any.whl Python 3 none any Details for tenthcheck-0.1.1-py3-none-any.whl

Total release size: 84.8 kB

Release files / tenthcheck-0.1.1.tar.gz

Download URL tenthcheck-0.1.1.tar.gz
Size 43.9 kB
Tags Source
SHA-256 checksum
How to use checksums
8014187ce5ca6392572b00bc2f522359bab36ce7f7c5b23714f9f8b650808f7d
BLAKE2b-256 checksum
How to use checksums
700cdcf2dfe2af17df68a2f0abe67e43083a70b16a6a7a948167f3ee6df5cc28
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.15.2

Release files / tenthcheck-0.1.1-py3-none-any.whl

Download URL tenthcheck-0.1.1-py3-none-any.whl
Size 40.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
98350e3b7fecf7ddf16284dc4cfba36c3d26ee4b70f74d0fd77276d2608c7d9f
BLAKE2b-256 checksum
How to use checksums
3d72ec9e20f7ce73c6db785bf44460892d44cb507c7287d62acfc3c0c4f22bde
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.15.2

Security for tenthcheck 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page