platitude

Detects AI slop by its rhetorical shape, not its vocabulary — and tells you exactly which sentence to fix and why.

Every public slop detector greps for words (delve, tapestry, em dashes) and dies on paraphrase. Slop survives paraphrase because it is not a vocabulary: it is a set of rhetorical moves — the manufactured contrast, the hollow reversal, the false profundity, the abstraction given a verb only a person can perform. platitude detects the moves, judges each one in context with a closed question, and combines that with a whole-text judgement into a verdict that is measured, not vibes: AUC 0.86 on a blind-labeled corpus where the strongest public tool scores 0.51 — a coin flip.

$ platitude "It's not a tool. It's a teammate. The future looks bright."
verdict: slop  (likelihood 67/100, model 96/100)

A2   not_x_but_y
     “It's not a tool. It's a teammate.”
     Interchangeable abstractions in a stock marketing reversal with no
     checkable content, followed only by vague boosterism.

---

Usage

Install

pip install platitude

Pulls torch for the surprisal layer. On a CUDA-less machine save ~2 GB with the CPU wheel first:

pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install platitude

The spaCy English model (~12 MB) downloads itself on first run.

Authenticate

The judge layer needs a Claude model, one of:

• ANTHROPIC_API_KEY in the environment (or a .env in the working directory) — fastest, ~2 s per call;
• a logged-in claude CLI on PATH — no key needed, runs on your subscription, ~10 s per call.

Verdicts are cached in ~/.cache/platitude/, so re-checking edited drafts only pays for what changed.

CLI

platitude "text to check"            # verdict + findings
platitude --file draft.md
platitude --json "text"              # machine-readable
platitude --no-l3 "text"            # free: candidate spans only, no model calls

Python

from platitude import verdict

v = verdict(open("draft.md").read())
v["verdict"]          # "slop" | "clean"
v["slop_likelihood"]  # 0-100
v["findings"]         # [{form, name, span, excerpt, reason}, ...]

Plug into an agent (MCP)

claude mcp add platitude -- platitude-mcp        # Claude Code

or in any MCP client config:

{"mcpServers": {"platitude": {"command": "platitude-mcp"}}}

One tool, check_slop(text). It returns the verdict and the confirmed spans with reasons, so the calling agent can rewrite the offending sentences instead of regenerating blindly. Typical loop: agent drafts → check_slop → agent fixes the named spans → re-check (cached, cheap).

Cost

One whole-text call, one bounded sweep call, plus one call per candidate span that survives the free filters — a tweet is ~2-4 calls, a 500-word post ~10-20. With the API that is a few cents per document.

---

How it works

The claim

Slop is a rhetorical shape, not a vocabulary. Word lists cannot see it, and worse, they cannot unsee it: real human writing uses the same shapes to carry actual content. A detector is only as good as its false-positive rate on exactly those texts. This project's central metric — which no public tool reports — is the hard-negative rate: of texts that contain a slop form but are good writing, how many get flagged?

Architecture

Four candidate layers overfire on purpose; judgement happens above them.

layer	what	forms
L0	vocabulary, punctuation, and density regexes	D group
L1	dependency-parse clause shapes (negation-then-assertion, staccato runs, agentless passives)	most of A, B
L2	surprisal at clause joints from a small local LM — a manufactured contrast's second half arrives too predictably	A1
L3	a frontier model answering one closed question per span: is this named form present here and doing content's work?	verdicts

Two verdict signals sit on top:

1. Whole-text sloppiness — one model call, the single strongest signal (AUC 0.824 alone).
2. Confirmed spans from the discriminative forms — contrast/reversal forms (A group), false-profundity forms (B group), and density tells (em dashes, quote density, stock pivots). Six forms whose confirmations measurably carry no verdict signal (filler adverbs, agentless passives, lazy absolutes among them) are detected but never judged and never scored — they were dropped by measurement, not taste.

A logistic combination (verdict-weights.json, leave-one-out-validated) turns both into the verdict; the confirmed spans double as the explanation. L3 never searches open-ended — a model asked to find problems in clean text finds them (measured: 0.317 hard-negative rate open vs 0.146 closed).

The corpus

196 English items, two independent label axes:

• forms — which shapes are present, with character spans. Objective.
• verdict — does the text read as slop? A judgement, and it does not follow from the forms. The corpus's whole point lives in the items where the axes disagree: 41 annotated hard negatives.

Nothing in the corpus was authored to be caught. Slop was collected: models asked to do what slop-posters do (three model families), plus wild pastes. Clean text is guaranteed-human pre-2021 Hacker News writing, the author's own posts and drafts. All verdicts come from one human labeling blind — shuffled, source hidden — and label stability was measured by a repeated blind pass: 93% self-agreement on the hardest items, which sets the ceiling any detector can reach here.

Results

Higher AUC = better ranking of slop below clean. Hard-FP = share of the 41 hard negatives flagged. 90% bootstrap CIs.

detector	auc	hard-fp	cost/text
platitude (shipped hybrid)	0.860 [.82, .90]	0.293	~half the full engine
whole-text model judgement alone	0.824 [.78, .87]	0.415	1 call
span engine alone	0.804 [.74, .86]	0.341	full spend
learned, free features only	0.676 [.58, .76]	0.195	0
Binoculars-style contrastive PPL (0.5B pair)	0.567 [.48, .65]	0.634	0
slop-guard (best public tool)	0.510	1.000	0

By length: the hybrid holds 0.85–0.87 on document-length items (catching 32–33 of 33 slop docs) and 0.85 on tweets. The residual errors are the measured taste boundary: gray-zone one-liners where even the labeler's own repeat-pass flips ~7%, and the labeler's own drafts, where personal voice tolerance defeats span counting.

Negative results, kept on purpose

Each of these looked right and was killed by a number:

idea	number that killed it
word lists (the incumbent)	AUC 0.510, flags 100% of hard negatives
A1 contrast from parse trees alone	0/4 recall, 13 false positives
normalising joint surprisal by content surprisal	0.602 vs 0.684 raw
judging every form the taxonomy names	six forms' confirmations carry zero-to-negative signal
prompt rules for the one-liner taste boundary	recalibration moved nothing outside CIs
contrastive perplexity as the verdict	0.567 at 0.5B / short text
free features + classifier as the verdict	0.676

Limitations

English only. One labeler — deliberately, since the tool's bar is its owner's, but generality is unproven. n=196 puts ±0.04–0.06 on every AUC; nothing here supports third-digit claims. Combination weights are fitted on this corpus (the design is LOOCV-validated; the shipped weights are the full fit). Verdicts require a frontier model; the judge model's own drift is uncontrolled. A concrete, checkable text in flawless AI cadence can pass — one known escape in 33 slop documents.

Reproducing

The measurement harness ships in the repo, not the package:

python3 bench.py                    # free detectors
python3 bench.py --detector hybrid  # the shipped configuration, LOOCV
python3 corpus_pool.py gen|hn|add   # grow the corpus without authoring
python3 label.py                    # blind verdicts; --repeat for stability
python3 build_corpus.py             # labels -> corpus.json

Adding a detector is a class with name, emits_spans, and run(texts) -> [{"score", "forms"}] registered in DETECTORS. No claim enters this README without a row from the harness.