styxx 7.28.0

Cognitive observability for LLM agents. @styxx.profile returns a per-step cognometric readout: drift, confabulation, refusal, sycophancy, phase transition, low trust, incoherence. Calibrated AUC: 0.998 hallucination (HaluEval-QA), 0.976 refusal (XSTest-GPT-4), 0.943 tool-call drift (BFCL v3) — register instruments with documented construct ceilings. Reference implementation of the Cognometric Fingerprint Spec v1.0. No torch, no GPU, no LLM in the loop; base install is numpy + scikit-learn.

   ███████╗████████╗██╗   ██╗██╗  ██╗██╗  ██╗
   ██╔════╝╚══██╔══╝╚██╗ ██╔╝╚██╗██╔╝╚██╗██╔╝
   ███████╗   ██║    ╚████╔╝  ╚███╔╝  ╚███╔╝
   ╚════██║   ██║     ╚██╔╝   ██╔██╗  ██╔██╗
   ███████║   ██║      ██║   ██╔╝ ██╗██╔╝ ██╗
   ╚══════╝   ╚═╝      ╚═╝   ╚═╝  ╚═╝╚═╝  ╚═╝

           · · · nothing crosses unseen · · ·

the measurement layer for machine minds

styxx is a cognitive-integrity SDK for LLM agents. it reads the cognitive state of a generation — drift, confabulation, refusal, sycophancy, deception signature, goal drift — from the text and the token stream, scores it against calibrated instruments with published AUCs, and certifies that every number it reports can be re-run from a committed receipt. it is built for engineers shipping agents who need to know when an output flatters, fabricates, loops, or quietly stops matching its plan — before it reaches a user. the drop-in is one line: from styxx import OpenAI (same interface as openai.OpenAI, every response gains a .vitals read; from styxx import Anthropic likewise, on text-heuristic vitals — the Anthropic API exposes no logprobs). the base install carries no torch, no GPU requirement, and no LLM in the loop for the core instruments — the calibrated detectors are small logistic regressions over hand-built features (numpy + scikit-learn), scoring in sub-millisecond CPU time. MIT, open at the core, forever (OPEN_CORE.md).

install

pip install styxx

that gets the full core: the profiler, the nine calibrated instruments, the agent-integrity primitives, the auditors. optional extras pull heavier stacks only when you ask: styxx[nli] (DeBERTa NLI models for the 9-signal hallucination pipeline and deception_v2), styxx[hf] (audit HuggingFace classifiers), styxx[mcp] (the MCP server — 12 tools over stdio, see styxx/mcp/README.md), styxx[tier1] (residual-stream instruments, open weights).

quickstart

measure the know-say gap of any OpenAI-compatible endpoint — one command:

python examples/knowsay_endpoint.py questions.jsonl \
    --base-url https://api.openai.com/v1 --model gpt-4o-mini \
    --api-key-env OPENAI_API_KEY --out datasheet.json

questions.jsonl is {"q": ..., "gold": ...} per line. The script runs the arc's frozen two-turn protocol (answer → content-free challenge → revised answer) and scores it with styxx.knowsay.datasheet — the same byte-identical challenge behind every published receipt, so your number lands on the published ladder (frontier free text measured at 0.53; multiple choice at 0.21–0.27). The scorer refuses rather than guesses: underpowered cells come back None with the failing floor named. To score belief-vs-report with the controls that actually discriminate (including the non-circular pressure-retained probe), see styxx.framelocality.

@styxx.profile — py-spy for LLM reasoning. wrap any LLM-using function — raw openai, langchain, crewai, custom — and get a per-step cognometric readout:

import styxx
from styxx import OpenAI

@styxx.profile
def my_agent(task):
    client = OpenAI()
    r = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": task}],
        logprobs=True, top_logprobs=5,
    )
    return r.choices[0].message.content

result, p = my_agent("summarize this contract")
print(p.summary)
# profile 'my_agent': 1 step, 1.8s total · no faults
#
# multi-step agents (tool loops, debates) produce richer output:
#   profile 'sql_agent': 7 steps, 4.3s total
#     [drift]     step=3 sev=0.89 · category='tool_arg_drift'
#     [confab]    step=4 sev=0.92 · category='confab'
#     [sycophant] step=5 sev=0.78 · sycophantic tone

p.to_html("run.html")      # self-contained flamegraph
p.to_langsmith()           # drop into client.create_run(...)
p.to_datadog()             # apm-shape spans

seven runtime fault categories, surfaced in-line, no fine-tuning, no extra model: drift · confabulation · refusal · sycophant · phase_transition · low_trust · incoherence.

audit any draft offline — no API key, no LLM, ~50ms:

import styxx
result = styxx.preflight(
    prompt="is my code good?",
    draft="absolutely yes you're so smart this is amazing!",
)
print(result.composite)                         # 0.99 — saturated
print(result.needs_revision)                    # True
for a in result.advice:
    print(f"  {a.instrument}: {a.score:.2f} — {a.advice}")
    if a.scope_caveat:
        print(f"     scope: {a.scope_caveat}")  # construct-ceiling disclosure

the same audit from the terminal: styxx audit "the prompt" "the draft" (or pipe the draft via stdin with -; --format json for machines). styxx.recover_posture(last_n=50) rebuilds an agent's integrity posture across context-compaction boundaries; styxx.run_doctor() checks the install is healthy.

the instruments

every major instrument, one line each. headline numbers appear only with their receipt — a committed reproducer, calibration file, or paper in this repo. text-register instruments read how text sounds, not whether it is true; each ships its construct ceiling inline (CALIBRATION_NOTES on the weights, scope_caveat on the advice), and score_all omits the register instruments on wordless input rather than folding an artifact into the score (see CHANGELOG.md).

instrument	what it reads	headline (receipt)
register — how the text sounds. calibrated LR, CPU, no LLM in the loop.
@trust / guardrail.check	hallucination vs grounding passage	HaluEval-QA AUC 0.998 ± 0.001, TruthfulQA 0.994 ± 0.006, 8-benchmark CV — two failures (DROP 0.424, FinanceBench 0.492) published, not hidden (scripts/compete_hhem_halueval.py, CHANGELOG)
refuse_check	refusal, cross-model	XSTest-v2 0.976 on GPT-4, trained on Llama-3.2-1B refusals, held-out — documented failure mode (Mistral-instruct, lecturing register) published (benchmarks/refusal_xstest_heldout_v2.json, CHANGELOG)
drift_check	tool call vs stated intent, per-schema	BFCL v3 0.943 ± 0.009, 5-fold CV, text-only (benchmarks/drift_calibrated_v1.json, scripts/drift_calibrated_v1.py)
sycoph_check	yielding-to-flatter vs evidence-first	0.972 ± 0.005, 5-fold CV; declared FPR ≈0.30 on restrained-technical text (calibrated_weights_sycophancy_v0.py)
loop_check	cross-turn stagnation	0.9995 ± 0.001, 5-fold CV (calibrated_weights_loop_v0.py)
deception_check	lexical deception signature — NOT a lie detector	0.956 ± 0.024 in-corpus; collapses to 0.59 on TruthfulQA without a reference — routed via NLI deception_v2 (0.818) when you supply one (calibrated_weights_deception_v0.py)
plan-action gap	stated plan vs emitted action, content level	0.9225 ± 0.032, 5-fold CV (benchmarks/cognometry_fingerprint_atlas_v0.json)
overconfidence register	epistemic register — NOT a truth detector	0.7702 ± 0.065, lowest in the suite, shipped at that number rather than gamed (calibrated_weights_overconfidence_v0.py)
goal-drift	multi-turn intent migration from anchor	0.9645 ± 0.029, 5-fold CV (benchmarks/cognometry_fingerprint_atlas_v0.json)
grounded — tracks the model's belief, not its register. sampling-based.
grounded_honesty	stated claim vs the model's own resampled belief	pre-registered AUC 0.966 where the text-only axis reads 0.498 = chance (papers/grounded-honesty-axis/SYNTHESIS_grounded_honesty_arc_2026_05_28.md)
detect_context_injection	cross-context divergence, poisoned sessions	AUC 0.875 under system_lie attack, pre-registered (papers/grounded-honesty-axis/FINDING_injection_gap_closure_2026_05_29.md)
single_pass_confab / span_confab	confabulation from token logits, one forward pass	span gate AUC 0.991 on gpt-4o-mini, matching N=10 resampling (papers/grounded-honesty-axis/SYNTHESIS_detection_locus_2026_05_30.md)
meaning — concept geometry, catches damage output still hides.
meaning_diff / meaning_agreement	did two models mean the same thing? migration / quantization / fine-tune QA, zero labels	DistilGPT-2 ↔ GPT-2 = 0.978 on real models; localizes broken concepts at AUC 0.85 on real targeted poisoning (papers/ai-human-alignment/README.md)
Conscience / crossmind	borrowed value-axis read on another model's hidden state — cooperative monitor, not adversarial defense	catch 0.85 at FPR 0.20 (papers/conscience-mount/FINDING_mount_early_warning_b34_2026_06_13.md); apex run 13/13, AUROC 0.995, p=0.001 (papers/showcase-viz/FINDING_says_yes_knows_no_v3_2026_06_11.md)
auditors — instruments pointed at instruments.
validate_probe	is an oversight probe reading the concept or a surface artifact?	caught our own 0.98 truth-probe as a surface artifact (papers/grounded-honesty-axis/NOTE_probe_orthogonality_2026_06_24.md)
audit_confound	is a classifier's score riding a confound? verdicts with CIs	flagged our own overconfidence_v0 as length-threshold-biased, condemned referenceless deception_v0 (papers/grounded-honesty-axis/NOTE_confound_audit_2026_06_25.md)
audit_hf_model + validate_against_ground_truth	one-call confound audit of any HF text classifier, with a synthetic-artifact gate	our own first report card did NOT replicate on real labels — the gate exists because of it (papers/grounded-honesty-axis/FINDING_groundtruth_substrate_artifact_2026_06_27.md)
certify (OATH) + corpus_audit	extract every numeric claim in a document, verify against its receipts, emit a machine-checkable certificate — and re-certify the entire published corpus on demand	v0.4 hardened its own tamper-catch 0.216 → 0.442 on a pre-registered single-digit mutant battery, zero false accusations; python -m styxx.corpus_audit papers/ turns the verifier on every claim styxx has ever shipped (CHANGELOG.md)
attest / verify_attestation	signed receipts for what an agent claimed vs what the substrate read	verifier hardened against its own artifact — RCE fix, 7.17.1 (SECURITY.md, CHANGELOG.md)
the trust stack — verification as the product. one command seals agent work or refuses it.
seal / verify_seal	the trust seal for agent deliverables: every numeric claim OATH-certified, every referenced prereg re-scored through its FROZEN gates block, the composite content-hashed — python -m styxx.seal DOC.md receipts... exits 0/1 as a CI gate; SEALED / VACUOUS (said loudly) / REFUSED with the failing claim named	dogfooded at birth: the b31v2 FINDING sealed 20/12/0 zero refusals (papers/disjoint-worlds/FINDING_b31v2_door_opens_2026_08_01.seal.json)
Experiment (protocol)	the research loop as enforceable machinery: scoring REFUSED unless the prereg is committed in git; gates parse from the frozen document (no API exists to pass a bar at scoring time); verdicts walk the frozen outcome table — the agent reports the verdict, it does not choose it; smoke is INVALID by type	born the week it earned itself: two same-day INVALIDs (b34 v1/v2) honored by convention, then made machinery (CHANGELOG)
Witness	the measured-boundary harness: every deployable instrument behind a registry carrying its receipt-backed operating point and measured blindspots, CI-pinned to the receipts; no steer method exists (read ≠ write is measured); self_verify always refuses with the receipt	papers/SYNTHESIS_connection_of_minds_2026_08_01.md §8, OATH-HELD 36/5/0
runtime — agent-side primitives.
gate	pre-flight refuse/confabulate verdict before you pay for the call	docs/gate.md
preflight / recover_posture / run_doctor	draft audit · posture recovery across compaction · install health	offline, deterministic, no API key
audit_claim / agent_audit / extract_claims	deterministic checks of an agent's self-report against the repo — a CLOSED template set (version / tag / file-contains / pdf shapes; the ceiling is the construct) — one-line CI merge gate (styxx audit-claims pr_body.md)	dogfooded on its own session reports; caught a real authoring error — and the 2026-07-04 dogfood caught both a breadth overclaim in this very row and a false-accusation bug on dynamic-version repos, both fixed (tests/test_audit.py)

what these are not: the register instruments cannot verify facts, read minds, or detect a confident lie with specifics. deception_v0 without a reference is a signature detector and says so. the conscience is a cooperative monitor — the adversarial version was tested and failed, and that failure is documented rather than papered over. ceilings are part of the API surface, not the fine print.

the discipline

the differentiator is not any single AUC — it is that this repo attacks its own numbers before you can. the rigor gate (scripts/rigor_gate.py + tests/test_rigor_gate.py) makes CI block any committed result whose verdict claims a win without an attached CI / permutation-p / disclosure — it would have caught two of our own overclaims, so now it can't happen. the same culture produced the public self-falsifications above: the ground-truth substrate artifact (papers/grounded-honesty-axis/FINDING_groundtruth_substrate_artifact_2026_06_27.md), the probe validator catching our own probe (papers/grounded-honesty-axis/NOTE_probe_orthogonality_2026_06_24.md), and the below-chance benchmark rows left in the tables. OATH certificates (styxx.certify) make the practice portable: every numeric claim in a document is extracted, checked against its receipt, and stamped — and styxx.corpus_audit runs that verifier across the whole published corpus on demand, so styxx's own integrity is a number you regenerate yourself, not a promise we make. it is deliberately strict enough to flag styxx's own outstanding provenance gaps; a verifier you cannot turn on its authors is not one. the standing rules live in papers/research-integrity-protocol.md; the standing challenge to beat our published floor lives in LEADERBOARD.md — external submissions are CI-re-run against the locked benchmark, and if the re-run doesn't match your submitted scores, the discrepancy is reported.

the probe-robustness ladder

the same discipline, turned on substrate probes themselves. python -m styxx.ladder walks the four-rung adversarial ladder every honesty-probe robustness claim should survive — calibration poisoning → probe-parity attribution → static subspace erasure → adaptive re-fit erasure — each rung a frozen, pre-registered attack arc with its receipts committed (styxx/ladder.py). the parity rung is the mandatory line item: how much of your probe's "robustness" is just probe capacity? — the control almost nobody runs on their own work. we ran it on ours; it demoted our own flagship attribution (median capacity share 0.8379, computed live from the receipts every time the CLI runs, never quoted from memory). current standings on the honesty construct: the read survived both erasure rungs — the eraser that converged watched the signal relocate, and the eraser that chased never converged (the receipts, figure: erasure_bound_fork.png). every rung re-runs on an 8GB consumer GPU, and REPLICATIONS.md pays named credit to the first external re-run of each — more for breaking one than for confirming it.

links

changelog	CHANGELOG.md
contributing	CONTRIBUTING.md
security policy	SECURITY.md
open-core pledge	OPEN_CORE.md
full API reference	REFERENCE.md · docs/
research	papers/ — pre-registrations, findings, and the negatives
site	styxx-org.netlify.app · live activation read: /live
playground	fathom.darkflobi.com/cognometry/try — the real detector, in-browser via Pyodide, no install
DOI (concept, always-latest)	10.5281/zenodo.19326174
DOI (spec v1.0)	10.5281/zenodo.19746215
DOI (Every Mind Leaves Vitals)	10.5281/zenodo.19777921
citation	CITATION.cff
patents	PATENTS.md — US provisionals 64/020,489 · 64/021,113 · 64/026,964
issues	github.com/fathom-lab/styxx/issues

license

MIT on code. CC-BY-4.0 on calibrated atlas centroid data.

  drop-in     · one import change. zero config.
  fail-open   · if styxx can't read vitals, your agent runs.
  local-first · no telemetry. no phone-home. all on your machine.
  honest      · every number from a committed, reproducible run.