Metadata-Version: 2.4
Name: platitude
Version: 0.1.3
Summary: Detects AI slop by its rhetorical shape, not its vocabulary. Verdict + the exact spans and reasons.
Author: Vlad Arbatov
License: MIT
Project-URL: Repository, https://github.com/vladzima/platitude
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: anthropic>=1.0.0
Requires-Dist: spacy>=3.8
Requires-Dist: torch>=2.0
Requires-Dist: transformers>=4.40
Requires-Dist: mcp>=1.2

# platitude

**Finds AI slop in English prose and points at the exact sentences to fix.**

Public slop detectors match word lists: `delve`, `tapestry`, em dashes. A
word list breaks under paraphrase, and it flags good writing that happens to
use a listed word. platitude reads structure instead. It locates the
patterns slop is built from ("it's not X, it's Y", the setup that exists to
be reversed, data that "tells us" things), asks a model one closed question
about each occurrence in context, and folds the answers into a whole-text
verdict. The result is measured: **AUC 0.86 on a blind-labeled corpus where
the best public tool scores 0.51, a coin flip.**

```
$ platitude "It's not a tool. It's a teammate. The future looks bright."
verdict: slop  (likelihood 67/100, model 96/100)

A2   not_x_but_y
     “It's not a tool. It's a teammate.”
     Interchangeable abstractions in a stock marketing reversal with no
     checkable content, followed only by vague boosterism.
```

---

## Usage

### Install

```bash
pip install platitude
```

Pulls torch for the surprisal layer. On a CUDA-less machine save ~2 GB with
the CPU wheel first:

```bash
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install platitude
```

The spaCy English model (~12 MB) downloads itself on first run.

### Authenticate

The judge layer needs a frontier model. First available wins:

1. `ANTHROPIC_API_KEY` in the environment or a `.env` in the working
   directory — fastest, ~2 s per call (claude-opus-5, the benched judge);
2. `OPENAI_API_KEY` — same protocol via the OpenAI API
   (`PLATITUDE_OPENAI_MODEL`, default `gpt-5.6-luna`);
3. a logged-in `claude` CLI on PATH — subscription auth, ~10 s per call;
4. a logged-in `codex` CLI on PATH — ChatGPT subscription auth.

Force one with `PLATITUDE_BACKEND=anthropic|openai|claude|codex`. The
numbers below were measured with claude-opus-5; other judges run the same
protocol but are unbenched — expect the ranking to hold, not the digits.

Verdicts are cached in `~/.cache/platitude/`, so re-checking edited drafts
only pays for what changed.

### CLI

```bash
platitude "text to check"            # verdict + findings
platitude --file draft.md
platitude --json "text"              # machine-readable
platitude --no-l3 "text"            # free: candidate spans only, no model calls
```

### Python

```python
from platitude import verdict

v = verdict(open("draft.md").read())
v["verdict"]          # "slop" | "clean"
v["slop_likelihood"]  # 0-100
v["findings"]         # [{form, name, span, excerpt, reason}, ...]
```

### Plug into an agent (MCP)

```bash
claude mcp add platitude -- platitude-mcp        # Claude Code
```

or in any MCP client config:

```json
{"mcpServers": {"platitude": {"command": "platitude-mcp"}}}
```

One tool, `check_slop(text)`. It returns the verdict *and* the confirmed
spans with reasons, so the calling agent can rewrite the offending sentences
instead of regenerating blindly. Typical loop: agent drafts → `check_slop` →
agent fixes the named spans → re-check (cached, cheap).

### Agent skill

For skill-based agents (Claude Code and compatible), ship the checking
discipline itself — when to check, how to read findings, when to stop:

```bash
git clone https://github.com/vladzima/platitude
cp -r platitude/skills/platitude ~/.claude/skills/    # global
cp -r platitude/skills/platitude .claude/skills/      # or per-project
```

### Cost

One whole-text call, one bounded sweep call, plus one call per candidate
span that survives the free filters — a tweet is ~2-4 calls, a 500-word post
~10-20. With the API that is a few cents per document.

---

## How it works

### The claim

Slop is a property of rhetorical shape. A word list misses every paraphrase
of a hollow move — and flags the same move where a human is using it to
carry actual content. A detector is only as good as its false-positive rate
on exactly those texts. This project's central metric — which no public tool
reports — is the **hard-negative rate**: of texts that contain a slop form
but are good writing, how many get flagged?

### Architecture

Four candidate layers overfire on purpose; judgement happens above them.

| layer | what | forms |
|---|---|---|
| L0 | vocabulary, punctuation, and density regexes | D group |
| L1 | dependency-parse clause shapes (negation-then-assertion, staccato runs, agentless passives) | most of A, B |
| L2 | surprisal at clause joints from a small local LM — a manufactured contrast's second half arrives too predictably | A1 |
| L3 | a frontier model answering one closed question per span: *is this named form present here and doing content's work?* | verdicts |

Two verdict signals sit on top:

1. **Whole-text sloppiness** — one model call, the single strongest signal
   (AUC 0.824 alone).
2. **Confirmed spans from the discriminative forms** — contrast/reversal
   forms (A group), false-profundity forms (B group), and density tells
   (em dashes, quote density, stock pivots). Six forms whose confirmations
   measurably carry no verdict signal (filler adverbs, agentless passives,
   lazy absolutes among them) are detected but never judged and never
   scored — they were dropped by measurement, not taste.

A logistic combination (`verdict-weights.json`, leave-one-out-validated)
turns both into the verdict; the confirmed spans double as the explanation.
L3 never searches open-ended: when the judge reviews a text with no fixed
question, it flags clean writing (measured: 0.317 hard-negative rate open
vs 0.146 closed).

### The corpus

196 English items, two independent label axes:

- **forms** — which shapes are present, with character spans. Objective.
- **verdict** — does the text read as slop? A judgement made on its own,
  decided separately from the forms axis. The corpus's whole point lives in the items where
  the axes disagree: 41 annotated hard negatives.

Nothing in the corpus was authored to be caught. Slop was *collected*:
models asked to do what slop-posters do (three model families), plus wild
pastes. Clean text is guaranteed-human pre-2021 Hacker News writing, the
author's own posts and drafts. All verdicts come from one human labeling
**blind** — shuffled, source hidden — and label stability was measured by a
repeated blind pass: **93% self-agreement on the hardest items**, which sets
the ceiling any detector can reach here.

### Results

Higher AUC = better ranking of slop below clean. Hard-FP = share of the 41
hard negatives flagged. 90% bootstrap CIs.

| detector | auc | hard-fp | cost/text |
|---|---|---|---|
| **platitude (shipped hybrid)** | **0.870** [.83, .91] | 0.293 | ~half the full engine |
| whole-text model judgement alone | 0.824 [.78, .87] | 0.415 | 1 call |
| span engine alone | 0.804 [.74, .86] | 0.341 | full spend |
| learned, free features only | 0.676 [.58, .76] | 0.195 | 0 |
| Binoculars-style contrastive PPL (0.5B pair) | 0.567 [.48, .65] | 0.634 | 0 |
| slop-guard (best public tool) | 0.510 | 1.000 | 0 |

By length: the hybrid holds 0.868 on document-length items (catching
32–33 of 33 slop docs) and 0.848 on tweets. The residual errors are the
measured taste boundary: gray-zone one-liners where even the labeler's own
repeat-pass flips ~7%, and the labeler's own drafts, where personal voice
tolerance defeats span counting.

### The hard rule

Since 0.1.3 the negation-then-substitution family — A2 ("it's not X, it's
Y"), A3 ("X isn't the problem, Y is"), A4 ("the question isn't whether…"),
A12 ("not just X…") — is flagged unconditionally, and a single occurrence
forces the slop verdict. Zero judge discretion, and content is no defense.
Quoted specimens and fenced code stay exempt as mentions rather than uses.

This is the owner's bar, set 2026-08-23 after contentful instances of the
construct kept reading badly in real contexts. The price is known and
accepted: measured against the corpus's original blind labels, the rule
converts 56 of 139 clean texts — several of the owner's own tweets among
them. The judge-gated configuration stays in the bench (`hybrid` row) as
the measured baseline; the shipped verdict applies the rule on top of it.

### Negative results, kept on purpose

Each of these looked right and was killed by a number:

| idea | number that killed it |
|---|---|
| word lists (the incumbent) | AUC 0.510, flags 100% of hard negatives |
| A1 contrast from parse trees alone | 0/4 recall, 13 false positives |
| normalising joint surprisal by content surprisal | 0.602 vs 0.684 raw |
| judging every form the taxonomy names | six forms' confirmations carry zero-to-negative signal |
| prompt rules for the one-liner taste boundary | recalibration moved nothing outside CIs |
| contrastive perplexity as the verdict | 0.567 at 0.5B / short text |
| free features + classifier as the verdict | 0.676 |

### Limitations

English only. One labeler — deliberately, since the tool's bar is its
owner's, but generality is unproven. n=196 puts ±0.04–0.06 on every AUC;
the third digit of every figure here is noise. Combination weights are fitted on
this corpus (the design is LOOCV-validated; the shipped weights are the full
fit). Verdicts require a frontier model; the judge model's own drift is
uncontrolled. A concrete, checkable text in flawless AI cadence can pass —
one known escape in 33 slop documents.

### Reproducing

The measurement harness ships in the repo, not the package:

```bash
python3 bench.py                    # free detectors
python3 bench.py --detector hybrid  # the shipped configuration, LOOCV
python3 corpus_pool.py gen|hn|add   # grow the corpus without authoring
python3 label.py                    # blind verdicts; --repeat for stability
python3 build_corpus.py             # labels -> corpus.json
```

Adding a detector is a class with `name`, `emits_spans`, and
`run(texts) -> [{"score", "forms"}]` registered in `DETECTORS`. No claim
enters this README without a row from the harness.
