Metadata-Version: 2.4
Name: platitude
Version: 0.1.0
Summary: Detects AI slop by its rhetorical shape, not its vocabulary. Verdict + the exact spans and reasons.
Author: Vlad Arbatov
License: MIT
Project-URL: Repository, https://github.com/vladzima/platitude
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: anthropic>=1.0.0
Requires-Dist: spacy>=3.8
Requires-Dist: torch>=2.0
Requires-Dist: transformers>=4.40
Requires-Dist: mcp>=1.2

# platitude

**Detects AI slop by its rhetorical shape, not its vocabulary — and tells you
exactly which sentence to fix and why.**

Every public slop detector greps for words (`delve`, `tapestry`, em dashes)
and dies on paraphrase. Slop survives paraphrase because it is not a
vocabulary: it is a set of rhetorical *moves* — the manufactured contrast,
the hollow reversal, the false profundity, the abstraction given a verb only
a person can perform. platitude detects the moves, judges each one in context
with a closed question, and combines that with a whole-text judgement into a
verdict that is measured, not vibes: **AUC 0.86 on a blind-labeled corpus
where the strongest public tool scores 0.51 — a coin flip.**

```
$ platitude "It's not a tool. It's a teammate. The future looks bright."
verdict: slop  (likelihood 67/100, model 96/100)

A2   not_x_but_y
     “It's not a tool. It's a teammate.”
     Interchangeable abstractions in a stock marketing reversal with no
     checkable content, followed only by vague boosterism.
```

---

## Usage

### Install

```bash
pip install platitude
```

Pulls torch for the surprisal layer. On a CUDA-less machine save ~2 GB with
the CPU wheel first:

```bash
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install platitude
```

The spaCy English model (~12 MB) downloads itself on first run.

### Authenticate

The judge layer needs a Claude model, one of:

- `ANTHROPIC_API_KEY` in the environment (or a `.env` in the working
  directory) — fastest, ~2 s per call;
- a logged-in `claude` CLI on PATH — no key needed, runs on your
  subscription, ~10 s per call.

Verdicts are cached in `~/.cache/platitude/`, so re-checking edited drafts
only pays for what changed.

### CLI

```bash
platitude "text to check"            # verdict + findings
platitude --file draft.md
platitude --json "text"              # machine-readable
platitude --no-l3 "text"            # free: candidate spans only, no model calls
```

### Python

```python
from platitude import verdict

v = verdict(open("draft.md").read())
v["verdict"]          # "slop" | "clean"
v["slop_likelihood"]  # 0-100
v["findings"]         # [{form, name, span, excerpt, reason}, ...]
```

### Plug into an agent (MCP)

```bash
claude mcp add platitude -- platitude-mcp        # Claude Code
```

or in any MCP client config:

```json
{"mcpServers": {"platitude": {"command": "platitude-mcp"}}}
```

One tool, `check_slop(text)`. It returns the verdict *and* the confirmed
spans with reasons, so the calling agent can rewrite the offending sentences
instead of regenerating blindly. Typical loop: agent drafts → `check_slop` →
agent fixes the named spans → re-check (cached, cheap).

### Cost

One whole-text call, one bounded sweep call, plus one call per candidate
span that survives the free filters — a tweet is ~2-4 calls, a 500-word post
~10-20. With the API that is a few cents per document.

---

## How it works

### The claim

Slop is a rhetorical shape, not a vocabulary. Word lists cannot see it, and
worse, they cannot *unsee* it: real human writing uses the same shapes to
carry actual content. A detector is only as good as its false-positive rate
on exactly those texts. This project's central metric — which no public tool
reports — is the **hard-negative rate**: of texts that contain a slop form
but are good writing, how many get flagged?

### Architecture

Four candidate layers overfire on purpose; judgement happens above them.

| layer | what | forms |
|---|---|---|
| L0 | vocabulary, punctuation, and density regexes | D group |
| L1 | dependency-parse clause shapes (negation-then-assertion, staccato runs, agentless passives) | most of A, B |
| L2 | surprisal at clause joints from a small local LM — a manufactured contrast's second half arrives too predictably | A1 |
| L3 | a frontier model answering one closed question per span: *is this named form present here and doing content's work?* | verdicts |

Two verdict signals sit on top:

1. **Whole-text sloppiness** — one model call, the single strongest signal
   (AUC 0.824 alone).
2. **Confirmed spans from the discriminative forms** — contrast/reversal
   forms (A group), false-profundity forms (B group), and density tells
   (em dashes, quote density, stock pivots). Six forms whose confirmations
   measurably carry no verdict signal (filler adverbs, agentless passives,
   lazy absolutes among them) are detected but never judged and never
   scored — they were dropped by measurement, not taste.

A logistic combination (`verdict-weights.json`, leave-one-out-validated)
turns both into the verdict; the confirmed spans double as the explanation.
L3 never searches open-ended — a model asked to find problems in clean text
finds them (measured: 0.317 hard-negative rate open vs 0.146 closed).

### The corpus

196 English items, two independent label axes:

- **forms** — which shapes are present, with character spans. Objective.
- **verdict** — does the text read as slop? A judgement, and it does *not*
  follow from the forms. The corpus's whole point lives in the items where
  the axes disagree: 41 annotated hard negatives.

Nothing in the corpus was authored to be caught. Slop was *collected*:
models asked to do what slop-posters do (three model families), plus wild
pastes. Clean text is guaranteed-human pre-2021 Hacker News writing, the
author's own posts and drafts. All verdicts come from one human labeling
**blind** — shuffled, source hidden — and label stability was measured by a
repeated blind pass: **93% self-agreement on the hardest items**, which sets
the ceiling any detector can reach here.

### Results

Higher AUC = better ranking of slop below clean. Hard-FP = share of the 41
hard negatives flagged. 90% bootstrap CIs.

| detector | auc | hard-fp | cost/text |
|---|---|---|---|
| **platitude (shipped hybrid)** | **0.860** [.82, .90] | 0.293 | ~half the full engine |
| whole-text model judgement alone | 0.824 [.78, .87] | 0.415 | 1 call |
| span engine alone | 0.804 [.74, .86] | 0.341 | full spend |
| learned, free features only | 0.676 [.58, .76] | 0.195 | 0 |
| Binoculars-style contrastive PPL (0.5B pair) | 0.567 [.48, .65] | 0.634 | 0 |
| slop-guard (best public tool) | 0.510 | 1.000 | 0 |

By length: the hybrid holds 0.85–0.87 on document-length items (catching
32–33 of 33 slop docs) and 0.85 on tweets. The residual errors are the
measured taste boundary: gray-zone one-liners where even the labeler's own
repeat-pass flips ~7%, and the labeler's own drafts, where personal voice
tolerance defeats span counting.

### Negative results, kept on purpose

Each of these looked right and was killed by a number:

| idea | number that killed it |
|---|---|
| word lists (the incumbent) | AUC 0.510, flags 100% of hard negatives |
| A1 contrast from parse trees alone | 0/4 recall, 13 false positives |
| normalising joint surprisal by content surprisal | 0.602 vs 0.684 raw |
| judging every form the taxonomy names | six forms' confirmations carry zero-to-negative signal |
| prompt rules for the one-liner taste boundary | recalibration moved nothing outside CIs |
| contrastive perplexity as the verdict | 0.567 at 0.5B / short text |
| free features + classifier as the verdict | 0.676 |

### Limitations

English only. One labeler — deliberately, since the tool's bar is its
owner's, but generality is unproven. n=196 puts ±0.04–0.06 on every AUC;
nothing here supports third-digit claims. Combination weights are fitted on
this corpus (the design is LOOCV-validated; the shipped weights are the full
fit). Verdicts require a frontier model; the judge model's own drift is
uncontrolled. A concrete, checkable text in flawless AI cadence can pass —
one known escape in 33 slop documents.

### Reproducing

The measurement harness ships in the repo, not the package:

```bash
python3 bench.py                    # free detectors
python3 bench.py --detector hybrid  # the shipped configuration, LOOCV
python3 corpus_pool.py gen|hn|add   # grow the corpus without authoring
python3 label.py                    # blind verdicts; --repeat for stability
python3 build_corpus.py             # labels -> corpus.json
```

Adding a detector is a class with `name`, `emits_spans`, and
`run(texts) -> [{"score", "forms"}]` registered in `DETECTORS`. No claim
enters this README without a row from the harness.
