Skip to main content

HarnessScope — turn-level governance observability for Claude Code sessions. Local-first: transcripts never leave the machine.

Project description

HarnessScope

HarnessScope — replay, circuit, rule health, token loss

Every frame above is hscope demo: a synthetic corpus, generated locally. Real transcripts carry full prompt text and are never demo material.

Turn-level governance observability for Claude Code sessions. HarnessScope parses your local transcript JSONL files into SQLite, judges every turn against your workspace's own operating rules (plan-before-code, verify-before-done, context budget, path boundaries, …), and serves a local web viewer that shows how each session traversed that governance structure — passes included, not just violations.

Screens: Session Replay (turn timeline with context chips + rule verdicts) · Harness Circuit (fixed-topology schematic of one turn's context sources → agent → action surfaces → governance outputs, with round-trip/off-host edges) · Rule Health (compliance × coverage per rule) · Token/Loop (per-turn token burn + loss pinpoint markers) · Setup (every hook, MCP server, skill and subagent you registered, next to what your transcripts show was actually called — active / unused / unregistered-call) · Plans hierarchy (work items above sessions — backend shipped, UI in progress).

Everything is local-first: no transcript data leaves your machine (the one exception — the opt-in Tier B LLM judge — is off by default and fires only on an explicit per-session click). The tool never writes to your projects, your .mcp.json or your harness settings: a tool that observes governance must not reshape the surface it observes. The single exception is hscope's own ~/.hscope/<workspace>/workspace-model.json, which the Setup page and the MCP server can edit — schema-checked, path-confined, and gated (a session token in the URL hscope opens; hscope --no-edit turns it off entirely).

Requirements

  • Python 3.11+ (3.10 works; developed on 3.13)
  • pip install -r requirements.txt — PyYAML, FastAPI, uvicorn. Nothing else.
  • No Node.js needed. The React UI ships pre-built in web/dist (committed on purpose); Node 20+ is only required if you want to hack on the UI itself (cd web && npm install && npm run build).
  • A machine with Claude Code transcripts under ~/.claude/projects/ (the default scan root).

Optional, only for the Tier B intent judge: anthropic (default backend, needs ANTHROPIC_API_KEY) or google-genai (HSCOPE_TIER_B_BACKEND=gemini, Vertex ADC).

See it without giving it your transcripts

hscope demo --serve      # builds a synthetic corpus, judges it, opens the viewer

Three invented sessions: one compliant, one wasteful (re-reads, dead context, a boundary read, a completion declared with nothing verifying it), and one fan-out (two subagents — one cites what it read, one does not; one spawned in the background — plus an off-host MCP round trip).

It is held to the same standard as real data: the corpus must satisfy hscope doctor, so the generator writing a shape the parser does not expect is caught by the invariants. It already earned that twice — a requestId reused across sessions (which hscope correctly read as one API call replayed by a fork), and a padded agentId: that the parser's regex swallowed whole.

Zero-config first run

pip install hscope       # or: pipx install hscope · npx -y hscope (Python 3.10+ still required)
hscope            # scan ~/.claude/projects → ingest → judge → check → open the viewer

No flags, no config file. Two things it will not get wrong:

  • The port is found, not assumed. 8000 is a reasonable default and on some Windows machines it is a reserved range (WinError 10013). A first impression should not be a stack trace.
  • It says that transcripts expire. The harness deletes them after about 30 days, and ingest is what makes them permanent. Ingest less often than that and you lose sessions silently — a session never ingested is indistinguishable from one that never happened. The first run prints how old your oldest transcript is and prescribes a cadence.

It also announces which rules it is about to judge with — neutral presets, or the workspace model(s) it found on disk. Claiming "presets" while quietly applying a model would be the same lie as a silent fallback.

Setup, delegated to the LLM you already have

Nobody should hand-write rules to try a tool. When the first run finds no workspace model, it seeds one from this directory's structure and leaves a page for your model to finish:

hscope                   # ...and, with no rules of your own, writes ~/.hscope/<id>/HSCOPE-SETUP.md
hscope mcp --install     # prints an MCP snippet; paste it into .mcp.json
hscope setup             # regenerate the page whenever your canon moves

HSCOPE-SETUP.md is one markdown page with two readers. For you: the canon, hooks, MCP servers, skills and subagents hscope found, and the canon clauses no rule cites yet. For your LLM: the instructions to call get_setup_guidesuggest_rulesset_rule and write those rules. You approve; it drafts.

The whole journey is three manual actions — run hscope, paste the snippet, hand over the page. The snippet carries HSCOPE_MCP_WRITE=1, so pasting it is the write consent; there is no second step. And hscope never edits .mcp.json itself: a tool that observes governance must not reshape the surface it observes.

Quickstart

pip install -r requirements.txt

# 1. Parse transcripts into SQLite (default scan: ~/.claude/projects)
python -m hscope.cli ingest --db harness_scope.db

# 2. Run the rule engine (add --model to overlay your workspace model)
python -m hscope.cli rules --db harness_scope.db \
    --model ~/.hscope/<workspace>/workspace-model.json

# 3. Serve the viewer (API + built UI)
HSCOPE_DB=harness_scope.db \
HSCOPE_WORKSPACE_MODEL=~/.hscope/<workspace>/workspace-model.json \
python -m hscope.server
# → http://127.0.0.1:8000

Steps 1–2 are idempotent — re-run them any time to refresh; step 2 wipes and rebuilds all derived judgments (rule evaluations, work-item links) from scratch.

Environment variables

Variable Default Meaning
HSCOPE_DB ./harness_scope.db SQLite path the server reads
HSCOPE_WORKSPACE_MODEL (unset) Path to workspace-model.json; unset ⇒ neutral built-in presets
HSCOPE_HOST / HSCOPE_PORT 127.0.0.1 / 8000 Server bind. Some Windows machines reserve 8000/8085–8184 (WinError 10013) — pick e.g. 9137
HSCOPE_WORKSPACE_ROOT repo parent Root the work-item convention's relative paths resolve against
HSCOPE_TIER_B_BACKEND / HSCOPE_TIER_B_MODEL anthropic / backend default Tier B judge provider/model (opt-in feature)
HSCOPE_MCP_WRITE (unset) 1 lets the MCP server edit its own workspace model. Unset ⇒ read-only

Sessions are not one model talking to itself

A session spawns subagents, and they run on models the parent does not. HarnessScope collects that structure instead of flattening it:

  • A subagent belongs to the turn that spawned it — and to the turn that resumed it. The harness stamps every record of a prompt with a promptId; a subagent's header record carries its parent turn's, and a SendMessage continuation carries the later turn's. So the link is read, never inferred from timestamps. 15% of this workspace's subagents were resumed at least once; their work is charged to the turn that asked for it.
  • A turn is one user prompt. Not "any user record with text": interrupt notices, <command-name> echoes and <bash-input> blocks are user records too, and counting them inflated turn totals by 28% on this workspace's own transcripts.
  • Each context window counts separately. A subagent reading a file the parent already read has not re-read anything — it never had those tokens. tool_uses.agent_id keeps the boundary, so re-read waste (R3/R8) is charged to the context that actually paid it. On this workspace that removed 42% of the reported re-read waste as false.
  • The model dimension survives. session_report returns which models ran, which of them only ever ran inside subagents, and what each spent.
  • The circuit draws the round trip. A subagent is a node with a spawn edge (who ran, on which model) and a return edge — labelled with what came back, which is not what the subagent spent. One real turn: three general-purpose agents on Opus burned 16.5k / 6.3k / 5.3k output tokens and handed the parent 1.5k / 2.2k / 2.0k. A background spawn's tool_result is only a launch acknowledgement, so its return size is reported as unmeasured rather than invented. Work a subagent did is tagged sub, never folded into the parent's.
  • One usage record per request — the last one. A streamed response writes several records for one requestId, and the early ones carry a partial output_tokens (2, where the finished response says 31,979). Deduplicating on the first occurrence undercounted this workspace's output by 3.6%, and far more inside subagents. (Two independent keys agree on the answer: max-per-message.id and last-per-requestId both give 32,348,242 where the naive sum of every record gives 90,957,147.)
  • A shared request is disclosed, not silently subtracted. A fork, resume or compact replays an assistant turn into a second transcript. Each session's own total is right; summing sessions counts that API call once per session. /api/summary reports shared_requests and shared_request_tokens (here: 28 calls, 27,059 output tokens, 0.08% of the total) so a workspace figure never pretends to be a clean sum.

The workspace model (optional but recommended)

Without a model, HarnessScope runs on neutral presets: generic rules with empty domain values, path labels fall back to file names, sessions render as a flat list. Crash-free in any repo — that's the de-domaining contract.

With a model (~/.hscope/<workspace>/workspace-model.json) you declare, in one JSON document, how your workspace is governed:

  • rules[] — archetype instances with your params (keywords, thresholds, markers)
  • roles[] — what each path / MCP server / subagent means (label, circuit node, host boundary)
  • gates[] — your governance pipeline (e.g. context-load → plan-approval → implement → verify → ledger)
  • unmeasurable[] — canon clauses no detector can check (surfaced honestly instead of faked)
  • work_items — optional upper grouping above sessions (this workspace: plans/** files with a status frontmatter; other repos can map issues/PRs into the same slot). Absent ⇒ flat sessions.

Schema and a neutral example live in hscope/schema/. Validate yours with:

python -m hscope.schema.validate_model ~/.hscope/<workspace>/workspace-model.json

MCP surface

Any MCP client can read what HarnessScope observed — and the loop closes: an agent opens a session by reading how the last one scored, instead of re-earning the finding.

// .mcp.json
{ "mcpServers": { "harness-scope": {
    "command": "python", "args": ["-m", "hscope.cli", "mcp", "--db", "/path/to/harness_scope.db"] } } }
Tool
list_sessions · session_report · token_hotspots · rule_health the viewer's numbers
get_workspace_model how this repo is governed, as one document
get_setup_guide · suggest_roles · suggest_rules evidence for authoring rules
set_role · set_rule write them back

No model runs inside this server. The client is the model: get_setup_guide hands it the closed archetype library and each archetype's param schema, suggest_rules hands it the canon clauses no rule cites, and the judgment comes back through set_rule — where the params are validated against that archetype's schema before anything is written. So the MCP path needs no provider, no credential and no off-host consent. hscope compile does the same job in one batch LLM call; this is its conversational, incremental counterpart.

Writes are off by default (HSCOPE_MCP_WRITE=1) and can only ever touch ~/.hscope/<workspace>/workspace-model.json. There is no tool that writes a project file or steers another agent — HarnessScope observes and judges; it does not drive.

Verification harnesses (Definition of Done)

Synthetic-transcript test suites, no fixtures required. Run them all with:

python -m hscope.verify_all   # discovers every hscope/verify_*.py; CI runs this on 3 OSes

Individually, for when you want one harness's report (this list is illustrative — the runner above is the source of truth, so a new harness never goes unrun):

python -m hscope.verify              # P1 parser (takes transcript paths; not in verify_all)
python -m hscope.verify_rules        # rule engine (archetypes, evaluations, de-domaining)
python -m hscope.verify_circuit      # circuit API (nodes, round-trips, gate lane)
python -m hscope.verify_work_items   # plan hierarchy (inference, lifecycle, guards)
python -m hscope.verify_context_tiers  # declared context-injection tiers
python -m hscope.verify_workspace    # workspace -> session model resolution
python -m hscope.verify_compile      # `hscope compile` (offline; stub provider)
python -m hscope.verify_defense      # parser defense / graceful degradation
python -m hscope.verify_mcp          # MCP surface (protocol, write gate, confinement)
python -m hscope.verify_agents       # multi-agent / multi-model collection (spawn turns, contexts)
python -m hscope.verify_doctor       # each invariant fires on its own break, and only its own
python -m hscope.verify_firstrun     # zero-config run (port discovery, retention notice)
python -m hscope.verify_demo         # the demo corpus: synthetic, ASCII, passes doctor
python -m hscope.verify_ingest       # incremental ingest: what must NOT be skipped
python -m hscope.verify_setup        # the handoff page + an LLM closing the loop over MCP
python -m hscope.verify_i18n         # viewer strings are English-canonical
python -m hscope.verify_scrub        # nothing publishable carries domain values

hscope doctor — drift is detected, not tuned away

The transcript format is undocumented and it moves: this corpus alone declares 27 harness versions in one month. hscope reads it anyway, because it is the only surface that knows meaning — which turn, which agent, which file. Metrics know quantities; a bill knows money; neither knows the word "subagent".

So the answer to drift is not endless tuning. It is a check that fails loudly, naming the harness version that broke it, instead of quietly emitting a wrong number:

python -m hscope.cli doctor --db harness_scope.db   # exit 1 if any invariant is violated

Nine invariants, each measured true before it shipped — a turn-start record carries its promptId; a subagent row carries its agentId; the counted usage row is the request's final usage; a request never spans a main and a subagent file; a subagent sits on a turn its parent really had; and so on.

Two things it refuses to do:

  • Judge rows an older hscope wrote. Afterwards, "the harness omitted the field" and "we never stored it" are the same absence. Sessions are stamped with the schema that read them (ingest_schema) and older ones are excluded and counted, never mixed in.
  • Report a vacuous pass. Zero violations over zero verifiable rows is not a green check.

Alongside them it prints metrics, never verdicts: the share of spawn calls that name the agent they started (skills and workflows spawn without one), the requests replayed into more than one session, and the share of assistant records that carry text while reporting ≤1 output token — the reason a per-agent spend is a floor rather than a measurement.

Failure behavior (graceful degradation)

A governance tool that quietly checks nothing is worse than one that stops. So:

Situation What happens
A transcript line will not parse Counted and reported as unparseable lines: N. Never dropped in silence.
A record type this build does not know Counted per type and printed (unknown rec types). Ingest continues.
A database written by a newer hscope Refused with an upgrade message. A column this build ignores is a rule that silently stops checking.
A database from before schema versioning Migrated forward and stamped.
--model points at a missing file Hard error. A typo must not quietly hand you the neutral preset's numbers as if they were yours.
A workspace model is unreadable, non-object, or from a future schema_version Refused by name. During auto-resolution the other models still load and each skip prints why.
$HSCOPE_WORKSPACE_MODEL is bad The viewer keeps serving with no roles, and says so once. (An empty circuit otherwise looks exactly like a compliant one.)
No LLM provider for hscope compile Deterministic structure + the neutral preset rules, with the provider's absence reported.
An MCP tool call is refused (unknown tool, bad params, writes off) Returned as isError content, not a protocol fault. The client reads the reason and keeps its session.
An MCP set_rule / set_role would produce an invalid model Nothing is written. A governance tool that corrupts its own config judges every later session against garbage.
A transcript has no promptId (pre-dates the field) Turns fall back to text heuristics, and the run reports how many turns were detected that way.
A subagent file names a spawn turn the main file never had Parked on turn 0 and listed as unlinked subagents. Silently parking it would charge a turn that spawned nothing.
A session is in the DB but its transcript is gone (~30-day retention) Re-ingest cannot rebuild it. It is counted and reported as not rebuilt, so a DB of two vintages never looks like one.
A subagent was spawned in the background Its returned payload size is unknown (the tool_result is a launch ack). The circuit says "background", never a fabricated token count.
A subagent is resumed under a prompt the main transcript never recorded Its rows keep the turn they were on, and the run reports unlinked resumes: N.

Data & privacy

  • The SQLite DB contains full prompt text from your transcripts. It is gitignored (*.db) and must stay local — never commit or upload it.
  • The server binds 127.0.0.1 by default and is read-only over the DB.
  • Tier B (LLM judge) sends a per-session digest off-host only when you click the judge button for that session; the rule (B1) ships disabled.

License & open-core boundary

HarnessScope is Apache-2.0 open core. Everything an individual user touches is free forever, with no feature limits — paid tiers (if/when they exist) only ever add things an organization needs, never remove things from individuals:

Layer Free (Apache-2.0 core) Enterprise (ee/, future)
Capture Transcript ingestion + hooks MCP gateway (cross-harness capture)
Analysis Rule engine · circuit · token loss · coverage — all of it Multi-user aggregation, org dashboards
Security Local security-flow rules (self-audit) Org compliance reports, audit logs
Accounts Local, single user SSO / RBAC

See LICENSE (Apache-2.0), ee/README.md (boundary), and CONTRIBUTING.md (DCO).

Repo layout

hscope/            # Python package: cli, parse (JSONL→SQLite), db, rules engine,
                   # work_items (plan hierarchy), server (FastAPI), mcp (stdio),
                   # compile (canon→model), tier_b, verify_*
hscope/schema/     # workspace-model JSON Schema + neutral example + validator
web/               # React viewer source (Vite); web/dist = committed build
docs/              # design notes per work package
ee/                # enterprise-edition boundary (empty placeholder — see ee/README.md)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hscope-0.1.1.tar.gz (262.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

hscope-0.1.1-py3-none-any.whl (292.7 kB view details)

Uploaded Python 3

File details

Details for the file hscope-0.1.1.tar.gz.

File metadata

  • Download URL: hscope-0.1.1.tar.gz
  • Upload date:
  • Size: 262.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.6

File hashes

Hashes for hscope-0.1.1.tar.gz
Algorithm Hash digest
SHA256 8111c04f001732068d3ea3f03cb9dad2d4051197e20fce8ce7386f569ee63f30
MD5 c78454179678c8f45a35e649ed070369
BLAKE2b-256 8c6bf041ac27b570f7d09dbaf1f40a5714ab71160d77fba2584fe08eafcda14f

See more details on using hashes here.

File details

Details for the file hscope-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: hscope-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 292.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.6

File hashes

Hashes for hscope-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 b9f769189446f8bda6dae89663d4bd4bb9093e10b12bf7001d5faaa62f9663eb
MD5 b81f55dcbf2e2294b7fb068a0fb6f74b
BLAKE2b-256 b80436501ce89ff5161ddfc7e7c08bed5239366c1ad4535d2d6de53e3c4ea3ba

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page