argus-agents 0.14.0

Catch silent failures in AI agent pipelines before production.

Your LangGraph pipeline runs fine — no exception. But three nodes later, something crashes with a KeyError. The real cause? A node upstream silently dropped a field. ARGUS catches this.

Beta, and under active development. ARGUS is early. Expect rough edges and bugs, and expect things to move. Issues and pull requests are welcome. Contributors: join the Discord before opening a PR — that is where updates land.

How to use ARGUS

1. Install

pip install argus-agents

2. Init

argus init

Writes .cursor/skills/argus-debug/ and .claude/skills/argus-debug/. Commit them. The skill already contains the setup prompt.

3. Attach

Ask your editor agent to wire ARGUS. (The skill already contains this AI setup prompt; the landing-page copy is just a fallback.)

from argus import ArgusRecorder
app = ArgusRecorder().attach(graph)

attach() returns the app you invoke. Nothing about your graph is patched or rewritten — ARGUS rides LangGraph's own callback stream. See Which entry point? if you are on the older ArgusWatcher.

4. Run

Same as always. Failures print in the terminal; clean runs stay silent.

[argus] run 8f3a1c02  silent_failure on retrieve
        missing: documents  (dropped by search)
        argus show last   |  argus ui

5. Inspect

argus show last
argus fix <id>     # paste-ready prompt for the root-cause node
argus ui

Empty dashboard → wrong directory or no run yet. Check project root or $ARGUS_DIR.

Optional — argus key set for the LLM judge. Skip it and you still get heuristics.

Optional — argus key set + one-line node purposes (ArgusRecorder(purposes={...}), or draft them with argus baseline --purposes) turns on the run reviewer: an LLM pass over the whole pipeline that confirms rule hits and catches hallucinated values and contradictions no rule can see.

Bring Your Own Key (BYOK)

AI-powered detection (the run reviewer, the semantic judge, LLM investigator, learned trends) uses your own key from the provider of your choice — OpenAI, Anthropic (Claude), or Google (Gemini). Set it once and it's saved locally for every future session:

argus key set                          # OpenAI by default — prompts, hidden input
argus key set --provider anthropic     # or Anthropic (Claude)
argus key set --provider google        # or Google (Gemini)
# pass it directly instead of being prompted:
argus key set sk-... --provider openai
# or just export it (env wins over the saved key):
export OPENAI_API_KEY=sk-...           # or ANTHROPIC_API_KEY / GEMINI_API_KEY

Configured more than one? Switch the active provider anytime:

argus key use anthropic  # activate a provider you already have a key for
argus key show           # list configured providers (masked); * marks the active one
argus doctor             # reports BYOK provider / hosted / heuristic-only mode

You pick the provider; ARGUS picks a sensible balanced model for each internal call (a cheap model for the frequent per-node checks, a stronger one for root-cause reasoning). Per-provider resolution order: env var (OPENAI_API_KEY / ANTHROPIC_API_KEY / GEMINI_API_KEY) → saved key → (hosted proxy, only with ARGUS_HOSTED_LLM=1: the prompts carry your pipeline's real values and pass through ARGUS servers) → heuristic-only.

No key? ARGUS still works — it falls back to heuristic-only detection, no crashes.

Hosted cloud sync is opt-in. argus login alone uploads nothing; full runs (every node input and output) go up only with ARGUS_CLOUD_SYNC=1. To keep a field out of the run file entirely, ArgusRecorder(redact_keys={"ssn", "api_key"}).

Telemetry is on by default and sends the shape of each run, never its data. Read Telemetry: shapes, never values before you run Argus on real data.

Telemetry: shapes, never values

Why it exists: we use these cards to measure which rules catch real failures and which fire on healthy runs. Without them, we can't tell whether Argus is right.

Example. Your node returns:

{"customer": {"email": "ana@acme.com"}, "refund_status": "scheduled"}

The card carries only this:

{"t": "object", "keys": {
  "k_91c2…": {"t": "object", "keys": {"k_4be0…": {"t": "string", "len": 12, "f": ["email_like"]}}},
  "k_e11d…": {"t": "string", "len": 9, "f": ["done_word"]}}}

• Values become shapes.
  • A string becomes its type, its length and a few features from a fixed list (empty, multiline, json_like, url, email_like, numeric, done_word, pending_word, error_word, refusal, placeholder). The features are computed on your machine, and the text itself is never sent.
  • A number becomes int or float, and its value is never sent.
  • A boolean keeps true/false, because ok: false is the signal we're measuring.
• Names become keyed hashes. customer becomes k_91c2…. The hash uses a secret stored only on your machine (~/.argus/telemetry.json) that is never sent, so nobody who reads the card can check a guess such as "is this customer?".
• What else is on the card: the verdict, the IDs of rules that fired (such as empty_output), which hashed node was blamed, node and edge counts, step timings, and the Argus, LangGraph and Python versions.

How you can check this:

• Only listed fields. The card is built field by field from a fixed list; no code copies part of the run. The server rejects any card with a field that isn't in the published schema.
• Canary tests. A test fills a pipeline with fake secrets and fails if any of them reach a card. It runs on every pull request.
• Read every card. Each card is written to .argus/outbox/ before it's sent. argus telemetry show-last prints the last one exactly as it was sent.

Turn it off. Any one of these works:

argus telemetry off      # this machine
export ARGUS_TELEMETRY=0 # this shell / CI job
export DO_NOT_TRACK=1    # every tool that honours it

argus telemetry status shows whether telemetry is on and why. Cards are deleted after 12 months.

Telemetry is separate from cloud sync: full runs, with real values, are uploaded only if you set ARGUS_CLOUD_SYNC=1. Every field and the full list of what is never sent: docs/DATA.md.

Quick Start (manual)

from argus import ArgusRecorder

app = ArgusRecorder().attach(compiled_graph)   # returns the app you invoke
result = app.invoke(initial_state)             # run is persisted automatically

ARGUS records every node, grades the run, and saves it. No changes to your node functions, and no changes to the graph either.

What it keeps is the dict each node returned — its update, before LangGraph merges it into shared state. That is the whole trick. A node that searches, throws the result away and returns {} leaves a full-looking merged state behind; only the update shows the silent no-op, and only then can blame land on the node that caused it instead of whoever crashes three steps later.

invoke, ainvoke, stream, astream and batch all work. One attach() serves many runs — each invoke, and each .batch() item, gets its own run file and its own verdict.

Which entry point?

ArgusWatcher keeps working and its docs below still apply to it. New code should use ArgusRecorder.

Declaring who reads what

A recording carries state, not code, so it cannot know that write three steps later needs the audience field plan produced. Tell it:

app = ArgusRecorder(consumers={"audience": ["write"]}).attach(graph)

Now a field that is never written, written empty, or written and then dropped fails on the node responsible — not on the node that happened to notice. Without a declaration ARGUS still catches empty updates, tool failures, crashes and degraded output; consumers= is what adds long-range field contracts.

A key can be a dotted path into nested state:

consumers={"email.body": ["send_email"]}

A top-level declaration means the whole value, so {"email": {"subject": "Re: order", "body": ""}} counts as a non-empty email — blanking a leaf is only caught if you declare the leaf.

Loops. Every visit of a reader is checked, not just the first. A field present on pass 1 and dropped before pass 2 fails on the node that dropped it. A field that only shows up by a later pass (filled in as the loop goes) is fine.

Two subgraphs with a node of the same name (two copies of one sub-agent, each with a retrieve) are kept apart as a:retrieve / b:retrieve, the subgraph name followed by the node name. Findings name them that way, and a consumer map must too ({"docs": ["b:retrieve"]}). Names that appear only once stay bare.

On a healthy run, argus consumers lists every later node that was handed a field and did not write it. Delete the nodes that only saw the field in shared state, then pass the file yourself. ARGUS does not load it, and does not fail CI on the guess.

It only sees what that one run did. A reader on a branch the run did not take is not listed, so record one healthy run per branch and merge the files. Fields you passed in as the graph's input are not listed either. Add them by hand when a later node depends on them.

argus consumers last --write argus.consumers.json

import json
from pathlib import Path

consumers = json.loads(Path("argus.consumers.json").read_text())
app = ArgusRecorder(consumers=consumers).attach(compiled_graph)

Fields that are legitimately empty. A PR review's issues: [] is the LGTM; a clean sanctions screen is hits: []. Declare them presence-only and absence alone fails:

consumers={"hits": {"readers": ["decide"], "allow_empty": True}}

That declaration also covers the tool responses of the node that writes the field, not just its own update — the OFAC tool returns {"hits": []} on a clean customer, and grading that critically failed every healthy KYC onboarding. A tool that raised, or a 4xx/5xx body, still fails hard. Undeclared empty retrieval lists keep the RAG default: critical.

Fields read on some paths only. draft_reply reads order when the ticket is a refund; on the FAQ path nothing writes it, and that is the design. Say so, and a field that was never written is fine, while one that was written and then dropped or blanked still fails:

consumers={"order": {"readers": ["draft_reply"], "required": False}}

Without that, "never written" blames the first step of the run, because a recording cannot say who was meant to write the field. That blame is a guess. When a node before the reader already failed, the guess lands on that node instead. With the run reviewer on, the guess only fails CI if the reviewer verified a problem on that step or on a node declared to read the field.

A final node is different: empty_output only fires when a node has a successor waiting, so a last node that returns {} is exempt by design (a terminal send_email legitimately returns nothing), and no later node reads its field. A healthy baseline closes that gap: a node that stops writing a key it writes on every healthy run fails.

Healthy baseline

Record what healthy runs write, per node, and pass it in. A later run that drops a key the node always writes, changes a field's type, or writes N/A / unknown / -1 where healthy runs held data then fails CI:

argus baseline <run-id> <run-id> --write argus.baseline.json   # one healthy run per branch

baseline = json.loads(Path("argus.baseline.json").read_text())
app = ArgusRecorder(consumers=consumers, baseline=baseline).attach(compiled_graph)

Only what every given run agrees on is kept, so a key written on one branch is not required on another. The file holds kinds (text, nonneg, object), never values. Without it those three checks are off; everything else runs.

Tools the callbacks cannot see

ARGUS files every tool call LangChain reports, and grades its payload as strictly as a node's own update. Under langgraph 1.x, code running inside a node function can hold an empty callback manager — so a tool the node invokes directly never fires on_tool_start, leaves tool_calls empty, and a 404 body or a raised exception has nothing to be found in. File it yourself:

from argus import report_tool_call

def fetch(state):
    try:
        hits = search_tool.invoke(state["query"])
    except Exception as exc:
        report_tool_call("search", input=state["query"], error=exc)
        raise
    report_tool_call("search", input=state["query"], output=hits)
    return {"hits": hits}

It writes the call in the exact shape the callback path uses, onto the node step currently open, so the graders cannot tell the two apart. The active recorder is resolved for you — pass recorder= or config= to be explicit. It never raises into your code: a call that cannot be placed logs one warning on the argus logger and returns False.

Async nodes on Python 3.9 / 3.10. Before 3.11, asyncio cannot hand LangChain's callback context to a child task. A tool awaited inside async def node(state) therefore fires no callback, and its errors are never graded. Pass the node's config on (async def node(state, config) … await tool.ainvoke(args, config)), call report_tool_call, or run 3.11+. attach logs a warning naming your async nodes when this applies.

ArgusWatcher (legacy path)

from argus import ArgusWatcher

watcher = ArgusWatcher()
app = watcher.attach(graph)         # StateGraph or already-compiled app
result = app.invoke(initial_state)

finalize() is optional. attach() wraps invoke() / ainvoke() / batch() / abatch() / stream() so the run is written to .argus/runs/ when the outermost call returns — including cyclic graphs. Calling watcher.finalize() afterwards is a no-op.

Constructor form still works if you compile yourself:

watcher = ArgusWatcher(graph)       # uncompiled StateGraph
app = graph.compile()
result = app.invoke(initial_state)

What It Catches

Detection Layers

Runs in order, each more expensive — only fires when needed. Every status a layer can assign, and how node statuses roll up into the run verdict, is specified in docs/STATUS.md.

1. Heuristics — 150+ failure signatures (placeholders, empty results, error keys, semantic degradation). Zero cost. On a node's own update, a status word (denied / declined / voided) or a findings list (errors: [...]) is warning-severity; a singular truthy error, bare success: False, and numeric HTTP status stay critical. Tool payloads are always graded critically. An empty lookup list fails unless declared allow_empty; a placeholder inside prose warns, except a template instruction ([INSERT …], [ENTER …], [YOUR …], [ADD …]) the node wrote itself, which fails.

2. Validators — custom per-node business-logic constraints. Deterministic.

3. Anomaly detector — statistical checks for output size anomalies, timing outliers. Deterministic.

4. Correlator — traces failure propagation across nodes. Points at the origin, not the crash site.

   Blame is decided against the state each node really saw, read off the recording. Where ARGUS has to reconstruct that state — it replays your reducers from a saved name, since a reducer callable cannot be stored in a run file — and its reconstruction disagrees with what the run actually recorded, the recording wins. A custom reducer (anything that is not operator.add or add_messages) is therefore approximated, never trusted over the trace, so a node whose [] your reducer discards is not reported as having dropped anything.

5. Whole-trace rules (argus.trace_rules) — read the finished run once, for what one step cannot show: the state schema, a healthy baseline, the model's raw output, every tool call in the run. Deterministic, critical, blamed on the step that caused it. A step that runs after another node already failed is not blamed again. Measured on a 255-fault suite: coverage 51% → 90%, with no false positive on 63 healthy runs, 54 of them real model prose.

6. Run reviewer (when node purposes are given) — reads the whole run once and verifies what looks wrong; a heuristic rule hit fails CI only when the reviewer agrees, and a rule warning it agrees with fails CI. Never clears a strict fail, never fails a step alone. See Run Reviewer. Without purposes, the LLM semantic judge runs instead: it reviews warning-level signatures the rules already raised, can dismiss a false-positive warning, and cannot fail CI or clear a hard rule fail.

7. LLM investigator — root cause explanations and debugging suggestions. Only on ambiguous failures.

Loop-Aware Inspection

Pipelines with loops (LLM -> compiler -> if fail, retry) get special treatment:

• Earlier rounds that self-corrected are marked retried (not counted as failures)
• Only the final round determines pass/fail
• Dashboard shows iteration badges, collapse/expand across attempts

A round, not a node name. Two Send workers running side by side are siblings, not a loop — they share a superstep. Each is graded on its own and none can relabel another, so a swallowed tool error in the first of five parallel workers still fails the build; previously the last worker's success buried it. An earlier round is only demoted to retried when every sibling in the final round passed.

A sequential loop still self-corrects as before: a ReAct agent whose first tool call 404s and whose second succeeds stays clean.

Replay

Fix a bug, re-run from the failing node. Skip upstream nodes entirely:

argus replay <run-id> node_7 --only --app mypkg.graph:build   # re-run that node against your graph
argus replay <run-id> node_7                                  # re-run from node_7 onward
argus diff <rerun-id>                                         # compare vs original

A rerun takes the state from the run file and the code from you. The input is the state node_7 really saw, rebuilt from the steps that already passed — they are never re-executed. The function comes from the compiled graph you pass with --app (a zero-arg callable returning it), so the fix you just made is what runs. ARGUS never goes looking for your source to import it.

Runs recorded the older way (ArgusWatcher) stored references to their own node functions and still replay without --app; those are labelled (legacy refs) in the header, and external API calls made during them were recorded to cassettes, so their replays are free and deterministic. On the trace path there are no cassettes — external calls execute live, and the command says so before it runs.

To grade a saved run with no graph at all, that's argus check <id>, not replay.

Time-Travel: edit the state, then resume

Spotted the bad value? Fix it in the saved state and resume from there — no code change, no re-running the steps that already worked:

argus replay <run-id> node_7 --set status=OK          # correct a value
argus replay <run-id> node_7 --delete stale_field     # reproduce a dropped field
argus replay <run-id> node_7 --patch fix.json         # a full patch document
argus replay <run-id> node_7 --set status=OK --dry-run  # preview, run nothing

The same rule applies: a trace run needs --app alongside these, --dry-run included — the graph is required before the patch is previewed. Upstream nodes stay frozen, so only the resumed trajectory changes. Paths are dotted with list indices — items[0].name — and match the field_path ARGUS reports on a failing signal, so you can paste one straight in. A patch file takes the same three ops:

{
  "delete": ["broken_field"],
  "set":    {"query": "fixed query", "meta.retries": 0},
  "merge":  {"config": {"temperature": 0}}
}

Patches are strict by default: a mistyped path errors with a "did you mean" hint instead of silently adding a field (use --create-missing to add new keys). Every patched replay records the patch it ran with, so the run explains its own divergence from the original.

Run Reviewer

Rules come in two kinds.

• Strict: nothing healthy produces them. A tool raised; a 5xx or an error body; {} with nodes waiting; a blank final answer; a cut-off generation that was used; a JSON-parse fallback; a regression against the healthy baseline. These fail CI on their own.
• Heuristic: usually a failure, sometimes the design. "I've approved a refund" matched against tool names; has_more on a web search (taking the top page is the design); a 404 that is the answer to "does this exist?" or a 409 "already exists" on an idempotent re-run; a number or ID the rule could not trace; N/A where the baseline had data; one tool call repeated (polling); a tool's requires_approval recorded as scheduled; "never written, so blame the first step".

Give ARGUS one sentence per node saying what it is for, and it runs a reviewer once per run. One model call reads the whole recording (each step's input, update, tool calls and model output) and lists what looks wrong. A second call checks each item against that step's own evidence only, must name the exact value the step should have written, and hands any arithmetic to code. What survives is verified.

Two independent checks must agree before a judgement call fails the build: a rule and the reviewer, or, where no rule can see the problem (paying 10× over the purchase order, the wrong vendor's bank account), two different models verifying the same item cold. The reviewer never clears a strict failure.

argus baseline <healthy-run> <healthy-run> --purposes --write argus.baseline.json

That drafts a purposes block next to the baseline. Edit it: it is what lets the reviewer tell "the reply is written before the refund runs, by design" from a defect. Then pass the file as usual:

baseline = json.loads(Path("argus.baseline.json").read_text())
app = ArgusRecorder(consumers=consumers, baseline=baseline).attach(compiled_graph)
# or: ArgusRecorder(purposes={"draft_reply": "Writes the customer reply ..."})
# review=False turns it off; review=True raises if there are no purposes

Everything it verified, gating or advisory, is listed under Run reviewer in argus show <run>. It runs when there are purposes and a key (or argus login). Without purposes it does not run: the same model with no purposes flagged about 40% of healthy runs.

Cost. One call per run plus one per reported item (at most 8), gpt-4.1 by default (mapped to your provider's capable model). It runs when the run finishes, so invoke() returns a few seconds later. Use it for CI and pre-deploy test runs, not on every production request. If any reviewer call fails, the run is graded by the rules alone. When the reviewer runs, the per-step judge below does not.

Semantic Judge

The rules fail the build. The judge reviews the flags they raised — that is all.

app = ArgusRecorder().attach(graph)                       # on when a key is set
app = ArgusRecorder(semantic_judge=False).attach(graph)   # rules only, fully deterministic

On by default once a provider key exists (argus key set). No key → off.

Judge last, never first

1. Rules (the cop) fail argus check. Empty {}, dropped field, HTTP 4xx, a tool that raised.
2. The judge is called only if those rules left a soft flag on that step (a warning-level signature — "this looks like a refusal"). Clean nodes are not asked. Hard fails are not up for debate.
3. If the judge says the flag is wrong, the flag is dropped. Customer text that reads like a refusal; an empty issues: [] that is a real LGTM.
4. If the judge agrees, the cop's answer stands. No second origin.

Walking every node looking for hallucinations is how a healthy pipeline went red on one run in a hundred, a different node each time. That path is closed.

Judging is also skipped for a turn that only issued tool calls — an empty content next to a populated tool_calls is how every tool-calling model works, and there is no prose there to rule on.

Every verdict carries an audit trail:

{
  "pass": false,
  "reason": "The output is completely unrelated to the input, which is about ingredients for a recipe.",
  "failure_kind": "unrelated",
  "confidence": 1.0,
  "evidence_considered": ["validator:payment_check", "anomaly:BA-003"],
  "overridden_signals": []
}

• failure_kind — unrelated / contradiction / empty_or_missing / other (audit only; none of these fail CI alone)
• evidence_considered — which prior signals the judge weighed
• overridden_signals — which it disagreed with and dropped

Silencing a noisy signature

A signature that keeps flagging legitimate output — say NL-002 reading the string "none" as a serialized null — should be silenced, not worked around by contorting your data:

argus ignore NL-002                    # everywhere in this project
argus ignore RF-001 --node draft_hook  # only on one node
argus ignore --list
argus ignore NL-002 --remove

Suppressions live in .argus/config.json (commit it so the team shares them). A suppressed hit no longer changes the node's status or the CI gate, but is still recorded on the run (suppressed_signals) so argus stats keeps counting it. argus doctor lists what's active.

Custom Validators

app = ArgusRecorder(validators={
    "classify": lambda o: (o.get("label") in ["yes", "no"], "unexpected label"),
    "*":        lambda o: ("error" not in o, "error key present"),  # runs on every node
}).attach(graph)

Validator failures cannot be overridden by the LLM judge — they are hard constraints.

Scored judges

A validator can ask for what it needs by parameter name and return a score instead of pass/fail. It declares any of output (the node's output), expected (your reference value) and metadata ({"node", "step_index", "run_id"}) and gets exactly those:

def total_close(output, expected):
    got, want = output["total"], expected
    return {"score": max(0.0, 1 - abs(got - want) / want), "label": "total", "explanation": f"{got} vs {want}"}

app = ArgusRecorder(
    validators={"calc": total_close},
    expected={"calc": 142},      # per node name, or "*" as the fallback
    judge_score_floor=0.5,       # a score under this is a warning (default 0.5)
).attach(graph)

A score is a number in [0, 1], or a dict with score and optionally label, explanation and direction ("maximize" by default; "minimize" for "lower is better"). Scores are recorded on the run: argus stats <run> lists them and argus diff <a> <b> shows the change per validator and node. A score under the floor is a low_judge_score warning finding. It never fails the node or argus check.

Everything else works as before: a validator that is not called by those names (below) is called with the output as its one argument, fn(output), and (ok, message[, severity]) still gates. Any other return from it, including a bare True / False, a string, None, or any bare number or score dict, is invalid and fails the node as it always did. A score only counts from a validator called by name. ARGUS calls a validator by name when, read from its own signature, every parameter it requires is output, expected or metadata (passable by keyword), and no positional parameter that is not passed by name comes before the first one that is. def judge(output, expected, threshold=0.5) is called by name (optional parameters after the names do not matter). These are still called fn(output) and never score: lambda out: ..., def v(state, expected=None), def v(x=None, output=None) (the output goes to x), a positional-only def v(output, /), def v(*args), and a builtin. A functools.partial keeps the keywords it pinned (partial(judge, expected="x") is given output only; one that pins every name it declares is called fn(output)). A functools.wraps wrapper is judged by its own signature, not the function it wraps, so a (*args, **kwargs) wrapper is called fn(output); set wrapper.__signature__ = inspect.signature(judge) to have it called by name. output and expected arrive as copies, so a judge cannot alter what is recorded. Judges are plain synchronous functions in your code; ARGUS does not load them from files, call an LLM for them, or time them out.

Configuration

ArgusRecorder takes its options as keyword arguments:

app = ArgusRecorder(
    validators={...},       # hard per-node checks (above)
    consumers={...},        # who reads which field, so blame lands on the writer
    purposes={...},         # one line per node: turns on the run reviewer
    baseline=baseline,      # from `argus baseline`: regressions against healthy runs
    semantic_judge=False,   # per-step judge (on by default when a key is set)
    strict=True,            # warning-level tool failures (e.g. HTTP 429) fail the step
).attach(graph)

ArgusConfig (timeouts, sampling, persistence) applies to the legacy ArgusWatcher:

from argus import ArgusWatcher, ArgusConfig

config = ArgusConfig(
    semantic_judge=True,           # review soft rule flags (default: on when a key is set)
    judge_model="gpt-4o",          # model for the judge
    node_timeout_ms=30000,         # flag outputs at ≥95% of this
    min_expected_ms=500,           # flag suspiciously fast LLM nodes
    sample_rate=0.5,               # persist 50% of clean runs (save disk)
    persist_failures=True,         # always persist failed runs
)
watcher = ArgusWatcher(graph, config=config)

CLI

argus list                           # all recorded runs
argus show last                      # most recent run
argus show <id>                      # inspect a specific run
argus check <id>                     # CI gate for an exact run; prints the JSON path checked
ARGUS_RUN_ID=<id> argus check        # CI-friendly selection when the id comes from an earlier step
argus check last                     # newest-file fallback — avoid in a shared workspace
argus check last --format json       # same verdict as one JSON object (run_id, overall_status, passed, findings[])
argus check last --fail-on crashed,silent_failure   # only these run statuses fail the gate
argus inspect <id> --step <node>     # dump raw input/output for a node
argus fix <id>                       # fix prompt for the root cause, ready to paste
argus replay <id> <node> --app m:fn  # re-run from a node, against the graph you pass
argus diff <id-a> <id-b>             # compare two runs
argus stats                          # signature hit stats, disable/enable/dispute signatures
argus ingest langsmith <file.jsonl>  # grade a LangSmith export — no app, no graph needed
argus edges mypkg.graph:build -o edges.json   # real topology for `ingest --edges`
argus ignore <SIG-ID> [--node N]     # silence a noisy signature project-wide or on one node
argus ignore --list                  # show active suppressions (.argus/config.json)
argus ui                             # web dashboard
argus doctor                         # check setup health + LLM mode (BYOK/hosted/heuristic)
argus key set [--provider ...]       # save a provider key locally (OpenAI/Anthropic/Google) — BYOK
argus key use <provider>             # switch the active provider
argus key show                       # list configured providers (masked); * marks active
argus key clear [--provider ...]     # remove one provider's key, or all
argus login                          # (optional) sign in for hosted cloud sync
argus logout                         # clear stored credentials
argus whoami                         # show current login status
argus update                         # check for newer release
argus telemetry status|on|off        # anonymous shape cards (docs/DATA.md)
argus telemetry show-last            # print the last card sent, exactly as sent

pytest plugin

Silent failures become test failures without changing how you invoke the graph:

pytest --argus

ARGUS records every LangGraph run in the test session through LangChain's public callback hook — nothing in LangGraph is patched or wrapped — and grades it exactly as ArgusRecorder().attach(graph) would. A graph you attached yourself is not recorded twice. A clean pipeline stays a passing test; missing fields, tool failures, crashes, and semantic degradation fail that test. Each test is graded against the run IDs saved during that test, so parallel pytest -n workers cannot grade one another's runs. Tests that never invoke a graph are unchanged. After a standalone CI run, pass its exact id with argus check <id> or ARGUS_RUN_ID=<id> argus check; argus check last only means the newest file and can select a stale or unrelated run in a shared workspace.

GitHub Action

Add the gate to any workflow, after your dependencies are installed:

      - uses: ArgusLabs-ai/ARGUS/.github/actions/argus-gate@v0.12.0
        with:
          pytest-args: tests/          # runs: pytest --argus tests/

The job fails when a run was not clean. To grade one saved run instead of your tests, pass run-id: (or set ARGUS_RUN_ID); it runs argus check <id>, never last. Other inputs: strict: warn_as_fail, version: "==0.12.0", install: "false" when your requirements already install argus-agents, working-directory.

Web Dashboard

argus ui    # opens at localhost:7842

Shows all runs, node-level detail, AI analysis, replay diffs, loop iteration badges, and comparison views. No account needed for local use.

If the table is empty, the UI is serving a different .argus than the project that just ran, or there are no runs yet. The empty state shows the path ARGUS is reading and what to do (argus show last, run the graph, check cwd vs project root).

• Distinct failure colors — crashed (red), silent failure (amber), semantic fail (purple), degraded input (orange), skipped (gray)
• Evidence audit trail — see exactly which signals the LLM judge considered and which it overrode
• Fat-trace step detail — every tool call a step made (input, output, and the error a tool raised even when the node swallowed it), plus what the run reviewer verified on that step
• Rerun from the dashboard — Rerun node / Rerun from here on a recorder run need the function that builds your graph (argus ui --app module:build_graph, or enter it when asked). A trace holds state, not code, and the dashboard never guesses where your code lives.
• Side-by-side diff — compare any two runs node-by-node

Without LangGraph

from argus import ArgusSession

session = ArgusSession()
session.set_edges({"fetch": ["classify"], "classify": ["process"]})

fetch    = session.wrap("fetch",    fetch_fn)
classify = session.wrap("classify", classify_fn)
process  = session.wrap("process",  process_fn)

state = fetch(initial_state)
state = classify(state)
state = process(state)
session.finalize()

Works with any framework — Prefect, Temporal, plain Python.

From a trace file, with no app at all

Already tracing to LangSmith? Grade the export directly — nothing imported, no graph, no rerun:

argus edges mypkg.graph:build -o edges.json        # real topology (optional but better)
argus ingest langsmith run.jsonl --edges edges.json --consumers consumers.json
argus check last

Tool and model child runs become the step's tool_calls / llm_usage. Without --edges, successors are guessed from step order; --consumers takes the same {"field": ["reader"]} map as ArgusRecorder(consumers=).

A skinny trace is refused rather than graded green: sampled runs, LLM-only spans, or payloads stripped at export leave nothing to detect a silent no-op with. ARGUS needs the dict each node returned. The one exception is a root run carrying an error — a graph that raised has no final state to export.

Requirements

• Python 3.9+ (3.11+ to capture tool calls inside async nodes without forwarding config)
• LangGraph 0.2+ (only for ArgusWatcher)
• A provider key (OpenAI, Anthropic, or Google) for semantic features — set via argus key set [--provider ...] (optional; all heuristic detection works without it)

For AI setup prompts and integration guides, visit arguslabs.in.

v0.14.0 — changelog

Contributing

See CONTRIBUTING.md. Join the Discord before opening a PR for updates and to talk through the change.

License

ARGUS is open-core. The open-source core (src/argus/, the argus-agents PyPI package) is licensed under Apache-2.0 — see LICENSE. The cloud/ directory (hosted/enterprise components) is proprietary — see cloud/LICENSE.