platitude

Finds AI slop in English prose and points at the exact sentences to fix.

Public slop detectors match word lists: delve, tapestry, em dashes. A word list breaks under paraphrase, and it flags good writing that happens to use a listed word. platitude reads structure instead. It locates the patterns slop is built from ("it's not X, it's Y", the setup that exists to be reversed, data that "tells us" things), asks a model one closed question about each occurrence in context, and folds the answers into a whole-text verdict. The result is measured: AUC 0.86 on a blind-labeled corpus where the best public tool scores 0.51, a coin flip.

$ platitude "It's not a tool. It's a teammate. The future looks bright."
verdict: slop  (likelihood 67/100, model 96/100)

A2   not_x_but_y
     “It's not a tool. It's a teammate.”
     Interchangeable abstractions in a stock marketing reversal with no
     checkable content, followed only by vague boosterism.

---

Usage

Install

pip install platitude

Pulls torch for the surprisal layer. On a CUDA-less machine save ~2 GB with the CPU wheel first:

pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install platitude

The spaCy English model (~12 MB) downloads itself on first run.

Authenticate

The judge layer needs a frontier model. First available wins:

1. ANTHROPIC_API_KEY in the environment or a .env in the working directory — fastest, ~2 s per call (claude-opus-5, the benched judge);
2. OPENAI_API_KEY — same protocol via the OpenAI API (PLATITUDE_OPENAI_MODEL, default gpt-5.6-luna);
3. a logged-in claude CLI on PATH — subscription auth, ~10 s per call;
4. a logged-in codex CLI on PATH — ChatGPT subscription auth.

Force one with PLATITUDE_BACKEND=anthropic|openai|claude|codex. The numbers below were measured with claude-opus-5; other judges run the same protocol but are unbenched — expect the ranking to hold, not the digits.

Verdicts are cached in ~/.cache/platitude/, so re-checking edited drafts only pays for what changed.

CLI

platitude "text to check"            # verdict + findings
platitude --file draft.md
platitude --json "text"              # machine-readable
platitude --no-l3 "text"            # free: candidate spans only, no model calls

Python

from platitude import verdict

v = verdict(open("draft.md").read())
v["verdict"]          # "slop" | "clean"
v["slop_likelihood"]  # 0-100
v["findings"]         # [{form, name, span, excerpt, reason}, ...]

Plug into an agent (MCP)

claude mcp add platitude -- platitude-mcp        # Claude Code

or in any MCP client config:

{"mcpServers": {"platitude": {"command": "platitude-mcp"}}}

One tool, check_slop(text). It returns the verdict and the confirmed spans with reasons, so the calling agent can rewrite the offending sentences instead of regenerating blindly. Typical loop: agent drafts → check_slop → agent fixes the named spans → re-check (cached, cheap).

Agent skill

For skill-based agents (Claude Code and compatible), ship the checking discipline itself — when to check, how to read findings, when to stop:

git clone https://github.com/vladzima/platitude
cp -r platitude/skills/platitude ~/.claude/skills/    # global
cp -r platitude/skills/platitude .claude/skills/      # or per-project

Cost

One whole-text call, one bounded sweep call, plus one call per candidate span that survives the free filters — a tweet is ~2-4 calls, a 500-word post ~10-20. With the API that is a few cents per document.

---

How it works

The claim

Slop is a property of rhetorical shape. A word list misses every paraphrase of a hollow move — and flags the same move where a human is using it to carry actual content. A detector is only as good as its false-positive rate on exactly those texts. This project's central metric — which no public tool reports — is the hard-negative rate: of texts that contain a slop form but are good writing, how many get flagged?

Architecture

Four candidate layers overfire on purpose; judgement happens above them.

layer	what	forms
L0	vocabulary, punctuation, and density regexes	D group
L1	dependency-parse clause shapes (negation-then-assertion, staccato runs, agentless passives)	most of A, B
L2	surprisal at clause joints from a small local LM — a manufactured contrast's second half arrives too predictably	A1
L3	a frontier model answering one closed question per span: is this named form present here and doing content's work?	verdicts

Two verdict signals sit on top:

1. Whole-text sloppiness — one model call, the single strongest signal (AUC 0.824 alone).
2. Confirmed spans from the discriminative forms — contrast/reversal forms (A group), false-profundity forms (B group), and density tells (em dashes, quote density, stock pivots). Six forms whose confirmations measurably carry no verdict signal (filler adverbs, agentless passives, lazy absolutes among them) are detected but never judged and never scored — they were dropped by measurement, not taste.

A logistic combination (verdict-weights.json, leave-one-out-validated) turns both into the verdict; the confirmed spans double as the explanation. L3 never searches open-ended: when the judge reviews a text with no fixed question, it flags clean writing (measured: 0.317 hard-negative rate open vs 0.146 closed).

The corpus

196 English items, two independent label axes:

• forms — which shapes are present, with character spans. Objective.
• verdict — does the text read as slop? A judgement made on its own, decided separately from the forms axis. The corpus's whole point lives in the items where the axes disagree: 41 annotated hard negatives.

Nothing in the corpus was authored to be caught. Slop was collected: models asked to do what slop-posters do (three model families), plus wild pastes. Clean text is guaranteed-human pre-2021 Hacker News writing, the author's own posts and drafts. All verdicts come from one human labeling blind — shuffled, source hidden — and label stability was measured by a repeated blind pass: 93% self-agreement on the hardest items, which sets the ceiling any detector can reach here.

Results

Higher AUC = better ranking of slop below clean. Hard-FP = share of the 41 hard negatives flagged. 90% bootstrap CIs.

detector	auc	hard-fp	cost/text
platitude (shipped hybrid)	0.870 [.83, .91]	0.293	~half the full engine
whole-text model judgement alone	0.824 [.78, .87]	0.415	1 call
span engine alone	0.804 [.74, .86]	0.341	full spend
learned, free features only	0.676 [.58, .76]	0.195	0
Binoculars-style contrastive PPL (0.5B pair)	0.567 [.48, .65]	0.634	0
slop-guard (best public tool)	0.510	1.000	0

By length: the hybrid holds 0.868 on document-length items (catching 32–33 of 33 slop docs) and 0.848 on tweets. The residual errors are the measured taste boundary: gray-zone one-liners where even the labeler's own repeat-pass flips ~7%, and the labeler's own drafts, where personal voice tolerance defeats span counting.

The hard rule

Since 0.1.3 the negation-then-substitution family — A2 ("it's not X, it's Y"), A3 ("X isn't the problem, Y is"), A4 ("the question isn't whether…"), A12 ("not just X…") — is flagged unconditionally, and a single occurrence forces the slop verdict. Zero judge discretion, and content is no defense. Quoted specimens and fenced code stay exempt as mentions rather than uses.

This is the owner's bar, set 2026-08-23 after contentful instances of the construct kept reading badly in real contexts. The price is known and accepted: measured against the corpus's original blind labels, the rule converts 56 of 139 clean texts — several of the owner's own tweets among them. The judge-gated configuration stays in the bench (hybrid row) as the measured baseline; the shipped verdict applies the rule on top of it.

Negative results, kept on purpose

Each of these looked right and was killed by a number:

idea	number that killed it
word lists (the incumbent)	AUC 0.510, flags 100% of hard negatives
A1 contrast from parse trees alone	0/4 recall, 13 false positives
normalising joint surprisal by content surprisal	0.602 vs 0.684 raw
judging every form the taxonomy names	six forms' confirmations carry zero-to-negative signal
prompt rules for the one-liner taste boundary	recalibration moved nothing outside CIs
contrastive perplexity as the verdict	0.567 at 0.5B / short text
free features + classifier as the verdict	0.676

Limitations

English only. One labeler — deliberately, since the tool's bar is its owner's, but generality is unproven. n=196 puts ±0.04–0.06 on every AUC; the third digit of every figure here is noise. Combination weights are fitted on this corpus (the design is LOOCV-validated; the shipped weights are the full fit). Verdicts require a frontier model; the judge model's own drift is uncontrolled. A concrete, checkable text in flawless AI cadence can pass — one known escape in 33 slop documents.

Reproducing

The measurement harness ships in the repo, not the package:

python3 bench.py                    # free detectors
python3 bench.py --detector hybrid  # the shipped configuration, LOOCV
python3 corpus_pool.py gen|hn|add   # grow the corpus without authoring
python3 label.py                    # blind verdicts; --repeat for stability
python3 build_corpus.py             # labels -> corpus.json

Adding a detector is a class with name, emits_spans, and run(texts) -> [{"score", "forms"}] registered in DETECTORS. No claim enters this README without a row from the harness.