Metadata-Version: 2.5
Name: stopslop
Version: 0.1.2
Summary: Measure a paper's prose against how a venue actually wrote, before and after LLMs existed
Project-URL: Homepage, https://github.com/dmitrii-khizbullin/stopslop
Project-URL: Source, https://github.com/dmitrii-khizbullin/stopslop
Author: Dmitrii Khizbullin
License: MIT License
        
        Copyright (c) 2026 Dmitrii Khizbullin
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: academic-writing,corpus,llm,prose,stylometry
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.9
Requires-Dist: pypdf>=5
Provides-Extra: pdfminer
Requires-Dist: pdfminer-six>=20240706; extra == 'pdfminer'
Provides-Extra: pymupdf
Requires-Dist: pymupdf>=1.24.3; extra == 'pymupdf'
Provides-Extra: pypdfium2
Requires-Dist: pypdfium2>=4; extra == 'pypdfium2'
Description-Content-Type: text/markdown

# stopslop

Measure a paper's prose against how a venue actually writes — before and after
LLMs existed.

Two baselines, always both: **fossil** (published up to 2022, so it cannot have
been machine-assisted) and **poisoned** (the last full year plus this one, which
cannot be assumed clean). The fossil window is the target; the poisoned window is
context, never an excuse — a field that moved because it is being written with
the same tools is not a neutral yardstick.

It measures and locates. It never edits your paper: a tool that strips tells on
the author's behalf produces text that passes a detector and still reads as
machine-written, so `check` reports where the prose sits and `locate` gives the
character offsets, and the writing stays yours.

## Install

The wheel carries the baselines for all seven venues — 105 venue-years of
measurements — so there is nothing to crawl and nothing to configure.

```bash
pip install stopslop              # pypdf, BSD-3-Clause, no copyleft
pip install "stopslop[pymupdf]"   # adds PyMuPDF (AGPL-3.0, opt-in)
```

Then, from any directory, in any project:

```bash
stopslop check paper.pdf --venue tmlr --out report.md
```

No system tools required. Everything else is stdlib `re` and `statistics`.

## Extractors

**The extractor is part of the measurement.** Across 24 papers the four
supported extractors agree to within ~1% on prose metrics — first person
18.1–18.3, -ly adverbs 13.2–13.4, passives 10.3–10.5 — and disagree sharply on
layout-sensitive ones: `numerals` runs **43.4** under pymupdf against **61.4**
under pypdf. So measurements are stored per extractor and never mixed, and
`stopslop check` refuses rather than compare across them.

| extra | licence | speed | notes |
|---|---|---|---|
| *(default)* `pypdf` | BSD-3-Clause | 0.82 s/paper | pure Python, installs anywhere |
| `[pymupdf]` | **AGPL-3.0** or commercial | 0.12 s/paper | best quality; opt-in, never a default |
| `[pypdfium2]` | BSD-3 / Apache-2.0 | **0.10 s/paper** | fastest; platform binary wheels |
| `[pdfminer]` | MIT | 2.13 s/paper | slowest |

Baselines for **all** of them ship in the base wheel, so an extra only decides
what you can *measure* with, never what you can *read*. Pick one with
`--extractor`.

`pip install stopslop` carries no copyleft: PyMuPDF is AGPL and is opt-in, so
taking on that licence is always your deliberate choice.

## Use

```bash
stopslop check paper.pdf --venue tmlr --out reports/paper.md   # score one paper against TMLR fossils
stopslop check paper.pdf --venue all --out reports/paper.md    # score one paper against combined fossils
```

`--venue all` is the default and the wider comparison: **1,033 papers in the
fossil window** against 20 to 226 for a single venue. More papers, and a broader
distribution, so fewer metrics stand out. Naming your venue is the sharper
comparison and the one to use when you know where the paper is going.

## Recrawl yourself

The shipped corpus is **read-only**: it is replaced by the next
`pip install --upgrade`, and a crawl landing among it would be indistinguishable
from it afterwards. So a corpus of your own is something you start on purpose:

```bash
stopslop init                                    # creates ./.stopslop/
stopslop crawl tmlr 2022 2025 2026 --limit 20    # fetch and measure
stopslop stats tmlr 2022                         # inspect one crawled year
stopslop where                                   # which corpus am I using?
```

```
.stopslop/pdf/<venue>/<year>/<paper-id>.pdf              the papers
.stopslop/stats/<extractor>/<venue>/<year>/stats.json    their measurements
```

Two subtrees, split by lifecycle: the PDFs are large, re-fetchable and
disposable — `rm -rf .stopslop/pdf` frees the disk without touching a number —
while the measurements are small and are what a report is built from. PDFs sit
above the extractor split because one copy of a paper feeds all four.

`.stopslop/` is found from the working directory or any parent, the way `git`
finds `.git`. **That is the only switch**: no environment variables, no flag, no
directory adopted because of its name. Everything that changes a number is
written down where a reader can see it.

**A corpus of your own replaces the shipped one; the two are never mixed.** Two
crawls made with different venues, limits and dates are two different samples,
and averaging them would put a number in your report that nobody could audit. So
you choose the venues, the years and the depth — and until you have crawled a
window, `check` says so instead of comparing against something thin. Delete
`.stopslop/` to go back to the shipped corpus.

`crawl` is the only command that touches the network, and it needs no key and no
account. Everything else is offline.

## Where the baselines come from

| | corpus | PDFs |
|---|---|---|
| **bundled** (default) | shipped in the wheel, read-only | none: a crawl is refused |
| **custom** | the nearest `.stopslop/stats/` | `.stopslop/pdf/` |

`stopslop where` prints which mode is in force and every resolved path, and every
report names its corpus in its header — so a number can always be traced to the
measurements it came from.

## Scope

**Seven venues today** — TMLR, NeurIPS, ICML, ICLR, ACL, EMNLP and JMLR — each
measured per year, with the fossil window reaching back to 2010 where the papers
exist. A venue is one entry in a registry, and `stopslop crawl` builds any of
them from scratch, so a corpus of your own is a command rather than a fork.

**Every number is auditable.** Each report prints the baseline it used, the
extractor, the date, and the n behind each window, and the same measurements sit
in plain JSON inside the package. `locate` gives the character offsets behind a
rate, so a claim about your prose can be checked against the sentence that
produced it.

**It compares prose to a corpus. It does not classify authorship.** A metric
outside the fossil range is a question about a sentence, not a verdict about a
writer — academic English is written well in many registers, and this tool
measures distance from one venue's historical distribution, nothing more.

## Analytics

The commands above score your paper. These read the corpus itself — how a metric
moved between the fossil and poisoned windows, per venue or across them — which
is where the baselines the checker uses come from.

```bash
stopslop trend tmlr --out reports/tmlr_trend.md                # one venue's drift
stopslop crosstrend --out reports/total_trend_points.md        # venues kept apart
stopslop pooltrend  --out reports/total_trend.md               # venues pooled
```

`crosstrend` keeps the venues apart, so agreement between them is visible — a
metric moving the same way in six of seven venues is a different kind of evidence
from one moving a lot in one. `pooltrend` merges them into a single pair of
windows.

Each of the three takes `--plot`, which renders the figure beside the table and
needs matplotlib. It is deliberately neither a dependency nor an extra: the
measurement path has no use for it, so install it yourself if you want figures.

MIT licensed.
