Metadata-Version: 2.4
Name: splent_feature_mail
Version: 1.5.0
License-Expression: LGPL-3.0-or-later
Requires-Python: >=3.13
Description-Content-Type: text/markdown
License-File: COPYING
License-File: COPYING.LESSER
Requires-Dist: splent_framework>=1.22.0
Dynamic: license-file

# splent_feature_mail

The `mail` feature of a [SPLENT](https://splent.io) product line.

Transactional mail over SMTP, for every other feature: `MailService`, an
outbox that records each mail with its state, its attempts and the last
error, retries in the background through the framework's task runner, a
screen for administrators with the mails that failed (`/admin/mail`), and
`splent feature:mail test` to check the settings. Standard library only
(`smtplib`, `email`), no third-party dependency.

A SPLENT feature is a Python package that a product composes with others.
Its models, routes, services, templates and hooks are registered when the
product starts. What this feature provides and requires is listed in the
`[tool.splent.contract]` section of `pyproject.toml`, which
`splent feature:contract --write` keeps in sync with the source.

## Sending a mail

```python
from splent_framework.services.service_locator import service_proxy

mail = service_proxy("MailService")
mail.send("ada@example.org", "Password changed", body)
mail.send(
    "ada@example.org",
    "Reset your password",
    body,
    html=html_body,
)
```

`send(to, subject, body, reply_to=None, *, html=None, sensitive=None)`
returns True when the mail went or is on its way, False when it could not
go. It raises nothing for a transport error, which is logged, recorded and
shown on `/admin/mail`. It raises `ValueError` for a header with a line
break and for an HTML part whose links are not the text's.

- `to` is an address or a list of them, `body` the plain-text part.
- `html` is the HTML part, sent as the text's alternative. It carries the
  same links as `body`, so a reader whose client shows one part only can
  follow every link; `send` refuses it otherwise, counting the links that
  differ and never quoting one. `html_from_text(body)` (also
  `MailService.html_from_text`) builds one from the text, escaped, with its
  paragraphs and its links, for a caller that has no HTML template yet.
- `sensitive` says whether a link in the mail carries a secret, a token
  that opens something. None, the default, takes any link to carry one;
  False says none does; True withholds the whole body, link or not (a
  one-time code).

## The outbox

Every mail is written to `mail_outbox` before anything is sent, on a
connection of its own and committed at once. It never waits for the
caller's transaction and never commits what the caller left pending, so a
mail is recorded whatever the caller does next, as it was sent whatever the
caller did next when `send` talked to the server itself.

| State | Means |
|---|---|
| `queued` | Waiting for an attempt, the first or a retry |
| `sending` | An attempt is under way |
| `sent` | The server took it |
| `failed` | Nothing more is tried: its attempts ran out, the server refused it for good (a 5xx answer, every recipient refused, no STARTTLS), or it carried a link and its one attempt failed |

A row keeps the recipients, the sender, the subject, both parts,
`retryable`, the `attempts` of its current round, `last_error` (the
exception's type and message, never a traceback) and when it was created,
last tried and sent.

### No body that carries a token

A link in a reset or confirmation mail opens an account, and a copy of the
outbox must open nothing. So what is kept depends on `sensitive`:

| The mail | Kept | Sent |
|---|---|---|
| No link, or `sensitive=False` | Whole, `retryable` | In the background by the task runner, retried |
| A link and `sensitive` left out | Every link cut to its site, `https://hub.example.org/[link removed]`, not `retryable` | Once, at once, by the process that holds it whole; `send` answers whether the server took it |
| `sensitive=True` | No body, a line saying it was withheld, not `retryable` | As above |

A mail that is not retryable is never sent again from the outbox, by the
task runner, the sweep or an administrator, since that would mean keeping
its token. Whoever waits for the link asks for a new one, which mints a new
token: the reset form, confirmemail's resend (at most one every
`CONFIRM_EMAIL_RESEND_SECONDS`, five minutes) and auth's
`/account/confirm-email`. The alternative, keeping what the mail is
rendered from and minting the token at send time, would make every caller
hand over a renderer instead of a body, and build its links outside the
request, where `SERVER_NAME` has to be set; it was left out.

### Retries

A retryable mail goes to `tasks.deliver` through `splent_framework.tasks`,
so `TASKS_MODE` decides where it runs: a thread of the web process (the
default), an RQ worker (`rq`) or the caller (`sync`, the tests). A failure
that may pass (a dropped connection, a timeout, a 4xx answer) is tried
again after 1, 2, 4, 8 and 16 minutes, six attempts over about half an
hour; a permanent one fails at once. An attempt claims its row (`queued` to
`sending` in one `UPDATE`), so two attempts never send it both. When the
runner cannot take the mail (`rq` with Redis down), one attempt is made in
the caller, and the sweep sends it later if that fails too.

In thread mode a retry waits in the web process, and a restart loses it.
The sweep, `splent feature:mail retry`, sends every mail queued and idle for
20 minutes and takes back an attempt that has not settled in 10 (its
process died), one attempt each, in the command, whatever `TASKS_MODE` says.
A process that dies after the server took a mail and before the row said so
leaves it to the sweep, and the mail may arrive twice: delivery is at least
once.

A database the migration has not reached yet has no outbox: `send` then
sends at once, as before, and logs it.

## For administrators

`/admin/mail`, in the Administration section of the menu (`mail_outbox`,
order 90, with the number of failed mails as its badge), shows the mails
that failed and the ones waiting for another attempt, with the reason. A
failed retryable mail can be sent again, which starts a new round of
attempts in the background, or discarded. One that carried a link is marked
so and has no "Send again".

## Commands

```bash
splent feature:mail test someone@example.org   # one mail now, and the server's answer
splent feature:mail retry                      # what lost its retries
splent feature:mail retry --failed             # ...and a new round for the failed
splent feature:mail purge                      # what is older than MAIL_OUTBOX_KEEP_DAYS
```

`test` prints where mail goes and how (never the password), sends a mail
with a text and an HTML part through the outbox, tried once in the command,
and exits 1 with the server's answer when it did not go.

`retry` and `purge` do not run by themselves. Schedule them on the host, as
the user the app runs as: the sweep every 15 minutes, the purge once a day.

```
*/15 * * * * docker exec -u splent <product>_web_deploy splent feature:mail retry
20 4 * * *   docker exec -u splent <product>_web_deploy splent feature:mail purge
```

Both exit with 0 when there is nothing to do.

## Configuration

| Variable | Default | Meaning |
|---|---|---|
| `MAIL_HOST` | `localhost` | The SMTP server. splent_feature_mailhog points it at its container in development |
| `MAIL_PORT` | 1025 | 587 with `MAIL_USE_TLS`, 465 with `MAIL_USE_SSL` |
| `MAIL_USE_TLS` | false | STARTTLS on a plain connection |
| `MAIL_USE_SSL` | false | TLS from the first byte. Not with `MAIL_USE_TLS` |
| `MAIL_TLS_VERIFY` | true | Whether either TLS checks the server's certificate and name |
| `MAIL_USERNAME` | empty | The account the app signs in with. Empty sends without signing in |
| `MAIL_PASSWORD` | empty | Its password, a secret: only in the deployment's `.env` |
| `MAIL_FROM` | `no-reply@localhost` | The sender. The admin panel's Sender setting wins when filled in |
| `MAIL_TIMEOUT` | 10 | Seconds each exchange with the server may take |
| `MAIL_OUTBOX_KEEP_DAYS` | 30 | Days a mail sent or given up on stays in the outbox |

`[tool.splent.config_schema]` types each one, all bound to the deployment,
and splent_framework 1.22 resolves them when the app starts: a port that is
not a number, a sender without an `@` or a password in the product's
`pyproject.toml` stops the start, naming the key. Then the feature refuses
what no product means:

- `MAIL_USE_TLS` and `MAIL_USE_SSL` together;
- `MAIL_TLS_VERIFY=false` outside development;
- in production, a `MAIL_USERNAME` with neither TLS nor SSL, which would
  send the password in clear.

Production on the development defaults (`localhost:1025`) is logged.

### TLS

Both TLS ways verify the server's certificate and its name with
`ssl.create_default_context()`. smtplib without a context verifies neither,
so whoever sat between the app and the server read the login and every
reset link. `MAIL_TLS_VERIFY=false` skips the check only in development
with splent_feature_mailhog installed; anywhere else the start refuses it,
and so does every attempt. MailHog itself speaks plain SMTP, which is what
splent_feature_mailhog sets (`MAIL_HOST`, `MAIL_PORT`, no TLS, no login).

## Use it in a product

```bash
splent feature:add splent_feature_mail
splent db:upgrade
```

## Develop

```bash
splent feature:test splent_feature_mail
splent feature:contract splent_feature_mail
```

## Release

```bash
splent feature:release splent_feature_mail --dry-run --bump minor
splent feature:release splent_feature_mail
```

The first command rehearses the release and changes nothing. The second
publishes it to the git forge and to PyPI, and a PyPI version is permanent.
