Metadata-Version: 2.4
Name: newsprint
Version: 0.11.0
Summary: Fetch starred newsletters over IMAP, reduce them to article content, and print them quarter-sheet, four to a side, duplex.
Keywords: email,imap,newsletter,pdf,printing,imposition
Author: Reuven Lerner
Author-email: Reuven Lerner <reuven@lernerpython.com>
License-Expression: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: End Users/Desktop
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Communications :: Email
Classifier: Topic :: Printing
Requires-Dist: anthropic>=1.4.0
Requires-Dist: beautifulsoup4>=4.15.0
Requires-Dist: click>=8.5.0
Requires-Dist: keyring>=25.7.0
Requires-Dist: lxml>=6.1.3
Requires-Dist: pillow>=12.3.0
Requires-Dist: pymupdf>=1.28.2
Requires-Dist: pypdf>=6.17.0
Requires-Dist: questionary>=2.1.1
Requires-Dist: wcwidth>=0.8.3
Requires-Dist: weasyprint>=69.0
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/reuven/newsprint
Project-URL: Repository, https://github.com/reuven/newsprint
Project-URL: Issues, https://github.com/reuven/newsprint/issues
Description-Content-Type: text/markdown

# newsprint

Turn a week of email newsletters into a small stack of paper.

newsprint fetches the messages you starred, strips each one down to its
article content, typesets it onto quarter-sheet cells, imposes four to a
sheet side, and prints it duplex — then unstars the mail and files it away,
so your inbox is clear and your reading is somewhere a screen isn't.

Star what you want during the week; run it once when you want the paper. It
was written for reading over Shabbat, away from screens, but nothing in it
is specific to that: it suits a flight, a commute, a weekend, or anyone who
would rather read long things on paper than on a phone.

## Requirements

- Python 3.12 or later
- [uv](https://docs.astral.sh/uv/)
- An IMAP account (including Gmail), and a folder you star newsletters into
- Pango, Cairo and gdk-pixbuf, for [WeasyPrint](https://weasyprint.org/) —
  see [Install](#install)
- macOS or Linux. Both are tested in CI on Python 3.12 and 3.14. Windows
  is not — see [Windows](#windows) for what stands in the way.
- A printer reachable via CUPS (`lp`), if you want newsprint to do the
  printing. With `--no-print` it hands you a PDF instead, which is a
  complete workflow on a machine with no printer configured at all.

## Install

Install it as a tool, which puts `newsprint` on your PATH so you never type
`uv run`:

```
uv tool install newsprint
```

Or, to try it without installing anything:

```
uvx newsprint --help
```

To install from a clone, run `uv tool install .` in the checkout. For
working *on* newsprint rather than with it, see
[Development](#development).

WeasyPrint needs Pango, Cairo and gdk-pixbuf, which are system libraries
rather than Python packages — install them first:

```
brew install pango                      # macOS
sudo apt install libpango-1.0-0 libpangoft2-1.0-0 libharfbuzz0b   # Debian/Ubuntu
```

### Windows

newsprint is developed and tested on macOS and Linux, and CI runs both. It
is **not tested on Windows**, and two things stand between it and a
working run there:

- **There is no `lp`.** Printing goes through CUPS, which Windows does not
  have, so `--no-print` is not optional — it is how you would always run
  it. `newsprint --no-print --output C:\Users\you\Reading` builds the
  packet and leaves you a PDF to print from whatever you normally print
  with. The imposition is already done, so print it at 100% ("Actual
  size", not "Fit to page") or the four cells land in the wrong places.
- **WeasyPrint's system libraries.** Pango, Cairo and gdk-pixbuf are not
  Python packages, and on Windows they come from the GTK3 runtime rather
  than from `pip` or `uv`. WeasyPrint's own [first steps
  guide](https://doc.courtbouillon.org/weasyprint/stable/first_steps.html)
  covers the Windows installation.

Nothing else is known to be in the way. The IMAP client is standard
library, the password lookup uses the Windows Credential Locker through
`keyring`, and the preview step does nothing rather than failing when
neither `open` nor `xdg-open` exists — the PDF's path is printed either
way.

That is a description of the obstacles, not a promise: nobody has run it
there. If you try it, an issue saying how far you got would be useful
whichever way it goes.

## Setup

The quickest way is to let newsprint ask:

```
newsprint --setup
```

It checks the credentials before writing anything, and lists your server's
own folder names to choose from — worth doing even if you would rather hand-
write the file, because the folder is the one setting nobody can guess from
outside. The hierarchy separator is not standardized (`.` on some servers,
`/` on others), and on Gmail a "folder" is a label.

To write the file by hand instead, copy the example config and fill in your mail account and printer:

```
cp config.example.toml ~/.config/newsprint/config.toml
$EDITOR ~/.config/newsprint/config.toml
```

The IMAP password is never stored in the config file. Put it in your system
keychain instead:

```
keyring set <mail.host> <mail.user>
```

for example:

```
keyring set imap.gmail.com you@gmail.com
```

See `config.example.toml` for the full set of options, including a note on
what Gmail specifically needs (an app password, and a folder path that isn't
`INBOX/toprint`).

## Choosing what to print

Star the messages you want, then run the tool. That is the whole workflow,
and it is not specific to any mail client: the tool never talks to your mail
program at all. It connects to your IMAP server and asks it for

```
UID SEARCH FLAGGED
```

`\Flagged` is a system flag defined by IMAP itself (RFC 3501), so whatever
you use to star a message — Thunderbird's star, Apple Mail's flag, Outlook's
follow-up flag, Gmail's star, your phone — is setting the same server-side
bit that this reads back. Star from anywhere; run this from your desktop.

Two things select a message: the **folder** narrows, the **star** picks. The
tool selects one mailbox (`mail.folder`) and searches for flagged messages
inside it.

How messages get into that folder, and how you decide what to star, is
entirely up to you — the tool only reads the result. The author's setup, as
one example: mail rules file every newsletter subscription into a `toprint`
folder as it arrives, and then during the week he stars the subset he
actually wants to read on paper. Everything unstarred stays in the folder
and is offered at run time. You could equally star straight from your phone
as things arrive, or star nothing during the week and pick entirely from the
checklist.

Anything in that folder you did *not* star is offered to you at run time, in
a checklist, so a newsletter you forgot to star is one keypress away rather
than a lost cause.

### Gmail

Gmail has no folders, only labels — but a label is exactly what IMAP shows
as a mailbox, so the workflow maps over cleanly:

| Other clients | Gmail |
| --- | --- |
| A rule moves newsletters into a `toprint` folder | A filter applies a `toprint` label |
| `folder = "INBOX/toprint"` | `folder = "toprint"` — a label is a top-level mailbox, not nested under INBOX |
| Star the ones to print | Star the ones to print |

You can also skip the label and set `folder = "INBOX"`, relying on stars
alone. That is less setup, but the "what else arrived this week" checklist
then offers your whole inbox rather than just newsletters.

Gmail also needs an app password rather than your account password; see
`config.example.toml`, which covers both points.

Two Gmail details worth checking on your first run, with `--no-retire` so
nothing is modified:

- **Superstars.** Gmail can show several star colors. They are widely
  reported to all map to the one `\Flagged` bit over IMAP, so "only red
  stars" is unlikely to survive the protocol — confirm before relying on it.
- **Retiring.** Messages are unstarred, marked read, and moved to the
  server's `\Trash` mailbox, which on Gmail is `[Gmail]/Trash`. That
  discovery is automatic, but worth watching once.

## Usage

```
newsprint
```

This fetches the starred messages, builds the imposed PDF, opens it in
Preview for a look, and asks before printing. Useful flags:

- `--dry-run` — build and preview the PDF, but print nothing and retire
  nothing. Exactly equivalent to passing `--no-print --no-retire` together;
  it exists as one flag because "show me what I would get" is a thing you
  want often enough to have a name.
- `--no-retire` — print for real, but leave mail untouched (the messages
  stay starred and will be reprinted next run). Useful for checking that a
  real printout looks right without consuming the print queue.
- `--output PATH` — write the finished PDF somewhere you can find it
  instead of a temp directory. An existing directory gets a dated and
  timed file inside it (`newsprint-2026-09-11-1432.pdf`), so building a
  second packet the same day sits beside the first rather than replacing
  it. Set `title` under `[packet]` and that name is used instead
  (`family-shabbat-reading-2026-09-11-1432.pdf`). Anything that is not an
  existing directory is taken as the file path to write.
- `--no-print` — build the PDF but do not send it to a printer; print it
  yourself from the file. Still offers to retire the mail, after asking
  whether the printing actually worked. `--no-print --output ~/reading/` is
  a complete workflow on a machine with no printer configured at all.
- `--unretire` — undo the last retirement: move those messages back out of
  the trash and re-star them. Builds nothing and prints nothing. See
  [Undoing a retirement](#undoing-a-retirement).
- `--no-preview` — skip opening the PDF in Preview.
- `--paper a4` / `--paper letter` — override the configured paper size for
  one run.
- `--cells-per-side 2` / `--cells-per-side 4` — override how many
  newsletters go on each side of a sheet. See [Bigger
  type](#bigger-type).
- `--since YYYY-MM-DD` — the first day the unstarred review should
  include, instead of letting the last successful run decide. Reach
  further back after a few weeks away (`--since 2026-08-01`), or narrow it
  to the last day or two. Starred messages are unaffected — those are
  always collected, however old.
- `--config PATH` — use a config file other than the default.

Run `newsprint --help` for the full list, and `newsprint --version` for the
release, project page and author.

### Bigger type

Four newsletters a side is the default: the sheet prints portrait and each
newsletter gets a quarter of it, A6 on A4 paper.

Two a side turns the sheet **landscape** and puts the two newsletters side
by side, so each one is A5 — portrait, twice the area, and the fold that
separates them runs down the middle of the sheet. That is the shape of an
ordinary booklet.

The extra area is *room* for larger type rather than larger type by itself:
at the same font size a bigger cell just makes the lines longer, and past
about 75 characters a line they get hard to follow. Measured on a real
packet:

| layout | `font_size_pt` | characters per line |
|---|---|---|
| four a side | 9 (default four a side) | 55 |
| two a side | 9 | 88 — too long |
| two a side | 12 (default two a side) | 66 |
| two a side | 14 | 55 |
| two a side | 18 | 44 — getting choppy |

So the type size follows the layout by itself — 9pt four a side, 12pt two
a side, both well inside the comfortable band. All you set is the layout:

```toml
[print]
cells_per_side = 2
```

or `--cells-per-side 2` for one run. Set `font_size_pt` under `[layout]`
only to override that, in either direction.

Half as many newsletters to a side means twice as many sheets for the same
reading, which is the trade.

## Reading from more than one folder

`folders` takes either one folder or a list of them:

```toml
[mail]
folders = "INBOX/toprint"
```

```toml
[mail]
folders = ["INBOX/toprint", "INBOX/work-reading"]
```

`folder`, the older singular name, is **deprecated**. It still works —
it is what every config written before this says, and nothing is going
to stop reading it — but a run that finds it says so, and offers to
rename the key for you. Accepting changes that one key and nothing else
in the file: not the value, not your comments, not a single other line.

Setting both is a config half-edited. `folders` wins, and newsprint says
which one it ignored rather than leaving you to notice a missing folder
in the packet.

Each folder is opened in turn on its own connection, and every message
remembers which one it came from — a uid is issued by a folder and means
nothing outside it, so two folders can both hold a uid 4 and they are
different messages. Retirement sends each back to its own folder, and so
does `--unretire`.

## Where packets go

With no `--output`, a finished packet is written to `[output] directory`
— `~/.local/state/newsprint/packets` by default — rather than to a temp
directory the system later deletes. That matters because a print run
retires the mail as soon as CUPS accepts the job, which is not the same as
paper having come out right: if it jams, the packet is still there to
print again.

Each run then removes packets in that directory older than `[output]
keep_days`, 30 by default. Set it to `0` to keep them all. The sweep only
ever touches PDFs directly in that directory, and never touches a
directory you named yourself with `--output`.

```toml
[output]
directory = "~/.local/state/newsprint/packets"
keep_days = 30
```

## Summary pages (optional)

newsprint can put one or two extra pages at the front of the packet, written
by Claude from the cleaned text of everything in it:

- **This Week's Topics** — what actually recurs across the week's reading.
  Not a list of subjects, which the contents page already gives you, but
  what connects them.
- **A page of your own** — only if you ask for one, by adding a
  `[summary.personal]` section. You describe in your own words what you are
  watching for in your reading, and give the page a heading; it flags things
  worth chasing. This was hardcoded to the author's own newsletter until
  newsprint became something other people install, which is why it is free
  text in your config rather than a setting.

**This is off by default, and stays off unless you turn it on.** With no
API key configured, nothing is sent anywhere and the packet prints exactly
as it otherwise would, with one line saying the summary was skipped and
why. There is no degraded mode and no silent failure: a missing key, no
network, an API error, or a timeout all end the same way — the packet
still prints, just without these pages.

To turn it on you need an [Anthropic API key](https://console.anthropic.com/).
The key never goes in `config.toml`. Point the config at a file that holds
it instead:

```toml
[summary]
enabled = true
```

That is the whole of it. Every other key has a default, so set one only to
change it:

| key | default |
| --- | --- |
| `api_key_file` | `~/.env` — a dotenv-format file |
| `api_key_var` | `ANTHROPIC_API_KEY` |
| `model` | `claude-opus-5` |
| `timeout_seconds` | `120.0` |

Optionally, add a page of your own. Leave this section out entirely and you
get only the topics page — its presence is the switch, so there is no flag
to set:

```toml
[summary.personal]
title = "Bamboo Weekly Candidates"
looking_for = """
I write a newsletter of pandas exercises built on real public datasets.
Flag things with public data plausibly behind them.
"""
```

Only `api_key_var` is read from that file, and the key is never written to
the run log or echoed on failure.

**What it costs, and what leaves your machine.** One API call per run,
carrying the cleaned text of every newsletter in the packet — on a typical
week that is roughly 40k input tokens and a few hundred out. That text goes
to Anthropic. If that is not something you want for your mail, leave the
feature off; everything else works without it.

## Images

Most images in a newsletter are decoration — a masthead, a social icon, a
tracking pixel — and on a quarter-sheet page they cost space the article
needs. So newsprint drops images by default, and keeps only the ones the
author treated as part of their argument.

An image is kept when it is at least 300px wide **and** the surrounding
text says it matters:

- the line before it ends in a colon, or reads like "here's the chart";
- the line after it opens a caption — `Source:`, `Chart:`, `Figure`,
  `Credit:`, `Data:`;
- the block after it is nothing but a parenthesised link, which is how
  Platformer cites the screenshots in "Those good posts".

Some newsletters are *built* out of charts and never introduce one in
words. Apricitas Economics sets fifteen to an issue: no lead-in, no
caption, `alt=""`. For those, three structural signs stand in for the
missing sentence, all three required:

- a paragraph of real prose on both sides — a figure inside an argument
  interrupts the argument, while a masthead sits above all of it and a
  footer logo below, with nothing on the far side;
- no `alt` **and** no `title` — which reads backwards and is not. A sender
  labels a masthead, a section header or a sponsor's logo precisely so it
  still reads as that brand when images are blocked; an author setting a
  chart for people to look at rarely bothers;
- a height in pixels rather than `auto`, since a content image carries its
  own dimensions through the mail template while chrome is laid out
  fluidly.

A kept image is fetched at print time, converted to grayscale, and resized
down to the cell's text column — 87mm at 200dpi, so 685px. A chart set on
a dark ground is turned over, so it prints as ink on white like the type
around it: white-on-black is a screen convention, and on paper it is a
page of toner. The test is the image's median grey level — Apricitas
Economics' charts measure 41 of 255 against an ordinary chart's 240, so
nothing real sits near the line. Nothing else is
ever fetched: a dropped image's URL is recorded for the run report and
never requested, which is also why tracking pixels never phone home.

Among the images that are *not* kept, one carrying substantial `alt` text
can leave a text placeholder in the flow instead, so the sentence around it
still makes sense:

```
[figure: US-China trade balances as a percent of GDP, 2010-2026]
```

In practice that is rare — most newsletter images ship with `alt=""`.

If an image cannot be fetched — a dead URL, a login wall, a timeout — the
run says so and prints the packet without it.

Measured across a 268-newsletter archive: **1.51 images kept per
newsletter**, and one text placeholder in the entire corpus. The
introduced-figure rule alone accounts for 0.97 of that; the uncaptioned
rule adds the rest, and it is what the chart-heavy publications live on —
Apricitas Economics goes from none of its charts to thirteen, and Prof G
Markets, Derek Thompson and David Skilling appear for the first time.

Charts cost paper. A packet that was seven sheet sides can be ten once the
charts are in it, and a publication that sets its charts on a black ground
prints as a black ground.

## Undoing a retirement

Retiring is the one irreversible thing newsprint does: printed messages are
marked read, unstarred, and moved to your trash. If a run retires something
it should not have, `newsprint --unretire` puts the last batch back.

```
newsprint --unretire
```

It reads the most recent retirement from the run log, finds those messages
in the trash **by Message-ID** — the uids the run recorded name nothing once
a message has moved — moves them back to the folder they came from, and
re-stars them. It asks before touching anything.

Two limits worth knowing:

- **It cannot outlive your trash.** Once the mail host empties it, the
  messages are gone and no run log will bring them back. Unretire soon or
  not at all.
- **Messages come back starred but read.** Retirement never records whether
  a message had already been read before the run, so that cannot be undone
  honestly, and is not guessed at.

Only the most recent retirement can be undone. If you need an older one, its
run log entry is in `~/.local/state/newsprint/runs/` and lists the
Message-IDs, which your mail client can search for.

## Filtering

`clean.py` removes only what it can confidently identify as chrome (an
"Unsubscribe" line, a "view in browser" bar, and the like); everything else
passes through. So an unsubscribe line, a mailing address, or a similar bit
of boilerplate can still appear on a printout if it doesn't clearly match
that check — `trim.py`'s final-cell judgment is the actual backstop against
a filler page, not `clean.py`'s removal.

## Development

The test suite runs on a clone with no extra setup, and reaches 100%
statement and branch coverage there:

```
uv sync
uv run pytest --cov=newsprint --cov-branch --cov-fail-under=100
```

A few tests need `tests/fixtures/` — a local archive of real newsletters,
built by `make fixtures` from the author's own mail. That directory is
gitignored, because it is other people's copyrighted writing and not the
author's to redistribute. Those tests assert corpus-wide properties ("no
real newsletter loses a headline to this rule") and skip cleanly when the
archive is absent; every line and branch they cover also has a unit test,
so the coverage guarantee does not depend on having them.


Rendering newsletters to PDF uses [WeasyPrint](https://weasyprint.org/), which
depends on Pango, Cairo, and gdk-pixbuf. On macOS, install them with Homebrew:

```
brew install pango cairo gdk-pixbuf
```

Homebrew's `lib` directory isn't on the default `dlopen` search path, so
WeasyPrint can fail to import with an error like `cannot load library
'libgobject-2.0-0'` unless something points the dynamic linker at it.
`newsprint._libpath` handles this automatically: before `render.py`
imports WeasyPrint, it sets `DYLD_FALLBACK_LIBRARY_PATH` to whichever of
`/opt/homebrew/lib` or `/usr/local/lib` has the library, if the variable
isn't already set. So running the tool, `newsprint`, or the test
suite all work out of the box on macOS with a Homebrew install in one of
those two locations — no manual export needed.

If your Homebrew libraries live somewhere else, set
`DYLD_FALLBACK_LIBRARY_PATH` yourself before running; an explicit value
always takes priority:

```
export DYLD_FALLBACK_LIBRARY_PATH="$(brew --prefix)/lib"
uv run pytest
```

Run the tests and linter with:

```
make test
make lint
```

## License

MIT. See [LICENSE](LICENSE).
