Metadata-Version: 2.5
Name: restverify
Version: 0.3.1
Summary: Rehearse and verify restic restores locally, on a schedule, with a diff-proof report.
Requires-Python: >=3.11
Provides-Extra: dev
Requires-Dist: build==1.6.1; extra == 'dev'
Requires-Dist: pytest==8.3.4; extra == 'dev'
Provides-Extra: keyring
Requires-Dist: keyring==25.5.0; extra == 'keyring'
Description-Content-Type: text/markdown

# restverify

[![PyPI](https://img.shields.io/pypi/v/restverify)](https://pypi.org/project/restverify/)
[![Python](https://img.shields.io/pypi/pyversions/restverify)](https://pypi.org/project/restverify/)

**A backup you haven't restored is a hope, not a backup. restverify restores
your restic snapshots and proves the data came back — every night, on exit
codes you can automate.**

restverify restores a snapshot to a private temp directory, compares it
against the original source tree, and delivers a verdict your shell can act
on:

| Exit code | Meaning |
|-----------|---------|
| `0` | restore verified |
| `2` | restored data differs from the source |
| `1` | the run could not complete (restic failed, no snapshots, restic missing) |
| `64` | usage problem — never a verification outcome |

Wire those to cron or CI and a silently degraded backup turns red long before
you need it for real.

restverify never creates backups, never runs prune or forget, never writes to
your source paths, and never stores or transmits your repository password.

## Why

`restic check` validates the repository's internal structure. It does not
prove a restore works — the restic project tracks this as open issue
[#2332](https://github.com/restic/restic/issues/2332): *“Backups are only real
if they're able to be restored.”*

The usual answer is a hand-rolled `restore && diff -r` script. It works the
day you write it, then decays silently: excludes drift, the diff gets slow,
someone comments out the cron line — and the first time it really runs is the
day you need the backup.

restverify is that script, done properly: one command, a recorded history,
a machine-readable JSON envelope, and a 442-test suite that is verified
against real `restic` repositories — including a 1.02 GiB / 1253-file drill
over an SFTP backend run as an unprivileged user
([transcript](https://github.com/niffyhunt/restverify/blob/main/docs/I9-DRILL.md)).

## Quick start

```bash
python3 -m pip install restverify        # or: pipx install restverify
```

Requires Python 3.11+ and a `restic` binary on PATH. restverify itself has
**zero runtime dependencies**.

```bash
# one-time: save the repository (and optional source path) to the config
restverify init -r /srv/backup -s /srv/data

# verify the newest snapshot
restverify run -r /srv/backup
```

```text
✓ restored snapshot b61d4bc1: 1253 files / 1.02 GiB in 12s; 0 diffs
```

That's the whole loop. `run` is read-only on the repository and never writes
to your source paths. Restores land in a private (`0700`) temp directory that
is removed on every exit — if the process is killed mid-restore, the next run
sweeps the leftovers. Nothing of the verification is left behind.

Look before you leap:

```bash
restverify run -r /srv/backup --dry-run
```

## What a run compares

With a source path configured (`-s` or saved via `init`), both trees are
walked with the same excludes and compared on:

- the set of entries (files, directories, symlinks)
- per-file size and kind, and symlink targets
- file/byte/symlink totals
- content digests over a deterministic sample

**The sample, precisely** (it is deterministic, so any run can be reproduced):
every file if there are ≤ 200, otherwise a stride of `ceil(total / 200)` in
sorted-path order; the largest file is always included, plus up to 50 files
matching your exclude patterns. Each sampled file is hashed with SHA-256, and
the run digest is the SHA-256 over the serialised `path ␀ size ␀ file-sha256`
lines in sorted-path order. The exact sampled list is in every `--json`
response, so you can re-verify any claim independently.

Not compared, on purpose: mtimes and ordering — they never fail a run.
Reported as warnings, promoted to exit 2 by `--strict`: an empty directory
present on only one side, and file mode/ownership drift.

Symlinks are recorded with their target and never followed (following one
could read outside the restore target). A path that was a file and came back
as a symlink is an error, not a silent read.

**Scope note:** without a source comparison (`--no-source`), a run proves
the restore *completed* — not that the data matches. The comparison is what
makes it verification; bring a source path when it matters.

## For machines: `--json`

```bash
restverify run -r /srv/backup --json > result.json
```

stdout carries exactly one JSON object and nothing else, in every mode —
success, dry-run, and every failure — so the redirect is always valid JSON.
Teaching text and errors go to stderr. The envelope is versioned
(`"schema": 1`); a breaking change bumps the schema rather than editing it
quietly. Errors carry a stable `kind` taxonomy
(`restic_missing`, `no_snapshots`, `restic_failed`, `source`, …) with a
human `what` and a `hint`.

The exit codes and the envelope are the entire integration surface: there is
nothing to scrape and nothing that changes under you between patch releases.

### Deliver the verdict: `--report-webhook`

```bash
restverify run -r /srv/backup --json \
  --report-webhook https://collector.example/hooks/restverify
```

After the run — pass, mismatch, or error — the exact JSON envelope is POSTed
to the URL (`Content-Type: application/json`, user agent
`restverify/<version>`), so a collector or pager sees the same object the
operator would have seen on stdout.

Delivery is best-effort by contract: a failure is one line on stderr and
**never changes the exit code** — the verdict is the verification's, not the
webhook's. Validation happens before anything runs (`https://` only, timeout
`0 < t <= 60` seconds, default 10; anything else is exit 64 and no network
attempt). Redirects are refused, credentials in the URL are never transmitted
or logged, and there are no retries — one attempt, one line of truth.

### Prove a snapshot: `prove`

```bash
restverify prove -r /srv/backup                # sample 10% of the newest snapshot
restverify prove -r /srv/backup --sample 100   # every file, plus a full seal check
restverify prove -r /srv/backup --seed 7       # a different deterministic sample
```

`prove` restores a **deterministic sample** of the snapshot's files and
verifies each one against restic's own `ls --json` record — and it adds a
second layer on top: `restic check --read-data-subset N%` over the **same
share** of the repository's data packs (`--sample 100` becomes a full
`--read-data`), which verifies the repository's cryptographic seals by
reading the data back.

Per-file limits, stated plainly: `restic ls --json` exposes no content
hashes (measured on restic 0.16.4), so the per-file claim is existence +
size only — `hashes_available` is `false` in the envelope. The content half
is covered by the seal check:
with one flipped byte in a real pack, `restic restore` exits 0 writing a
0-byte file (the size check catches that one) while `check` reports the
corruption — `prove` catches both.

Sampling is reproducible: same file list, same percent, same seed → same
sample. The default seed derives from the snapshot id, so re-running prove
on an unchanged snapshot re-checks the same files. The largest file is
always included — the likeliest corruption canary is never left to chance.

Exit codes are the standing contract: 0 all verified, 2 at least one sampled
file missing or size-differs **or** the seal check failed, 1 restic
failures (including a locked repository — that is could-not-complete, not
corruption), 64 usage.

### Browse the history: `dashboard`

```bash
restverify dashboard                # prints a http://127.0.0.1:PORT/ URL
restverify dashboard --port 8765    # a fixed port instead of a random one
```

One stdlib web page, read-only end to end: the newest 50 runs (status, exit
code, repo, snapshot, duration) plus a pass/mismatch/error summary. The
history database is opened with SQLite `mode=ro` — never created, never
written. No framework, no JavaScript, no form, no external resources.

Built-in defenses, each pinned by a test: binds `127.0.0.1` (a random free
port unless `--port`); `--bind-all` is refused (exit 64, before any socket
exists) unless paired with `--yes-i-know`; the `Host` header must name the
bound address, so a DNS-rebinding page cannot aim a browser at it; every
database value is HTML-escaped; responses carry `CSP: default-src 'none';
style-src 'unsafe-inline'`, `nosniff`, and `Cache-Control: no-store`; GET
and HEAD on `/` only — other paths 404, other methods 405; request logs
never repeat query strings. Ctrl-C exits cleanly.

## On a schedule

```bash
restverify cron -r /srv/backup            # one ready-to-paste crontab line
restverify cron -r /srv/backup --systemd  # a .service + .timer pair instead
```

`cron` **prints and never installs**: no crontab is modified, `systemctl` is
never called, nothing is written into any unit directory. You stay in control
of your own scheduler.

One note for scheduled runs: a scheduler runs with a minimal environment, so
set `RESTIC_PASSWORD_COMMAND` (or `RESTIC_PASSWORD_FILE`) in the unit or
crontab itself. restverify never stores your password.

## Sandboxed restores

```bash
restverify run -r /srv/backup --sandbox
```

The restore executes inside a disposable container instead of directly on the
host: your own `restic` binary is copied in, your temp dir is bind-mounted,
and the process runs as *your* uid — restored files are owned by you, never
root. Local repositories mount read-only, and the container has no network.
Remote repos (`sftp://`, `rclone:`) resolve inside the container with your own
credentials; a `password_command` that reads a host file works via
`RESTVERIFY_SANDBOX_PASSFILE` (mounted read-only).

The container is purged on every exit — including `kill -9` mid-restore;
containers carry an ownership label, and any orphan from a killed run is
swept automatically by the next run. Requires `docker` (or a compatible
runtime) on PATH with socket access for your user — never sudo. The gate was
drilled against real Docker: a clean sandboxed run, a SIGKILL mid-restore,
and proof the next run swept the orphan and finished with zero leftovers
([transcript](https://github.com/niffyhunt/restverify/blob/main/docs/I11-SANDBOX.md)).

Verification, history and exit codes are identical with or without `--sandbox`.

## The nightly CI drill

The restore drill ships as a reusable GitHub Action in this repository, so
your backup repo can drill itself on a schedule:

- [`.github/actions/restore-drill/`](https://github.com/niffyhunt/restverify/blob/main/.github/actions/restore-drill) — installs
  restic, downloads the **released** restverify wheel pinned to a version,
  verifies its GPG signature against the release key shipped inside the
  action, and runs two drills: a healthy fixture must exit 0, and a tampered
  fixture (one flipped byte) must exit 2.
- [`templates/ci-drill/`](https://github.com/niffyhunt/restverify/blob/main/templates/ci-drill) — a ready-made scheduled
  workflow (point it at your restic repo via secrets) and `drill.sh`, the
  same drill as a standalone script for plain cron, systemd timers, or any
  other CI.

A drill that can't fail is decoration: the tampered-fixture leg exists so a
nightly run that "succeeds" at everything proves nothing. Copy the template,
add your secrets, and a degraded backup turns the job red before you ever
need it for real. The drill runs the same outside GitHub as a plain script —
no GitHub required.

## Signed releases

Every release is signed; verification needs no keyserver:

```bash
# verify the tag
git tag -v v0.3.0

# verify a wheel/sdist from PyPI against the in-repo release key
curl -O https://files.pythonhosted.org/packages/<path>/restverify-0.3.0-py3-none-any.whl
curl -O https://files.pythonhosted.org/packages/<path>/restverify-0.3.0-py3-none-any.whl.asc
gpg --import docs/release-key.asc
gpg --verify restverify-0.3.0-py3-none-any.whl.asc restverify-0.3.0-py3-none-any.whl
```

Release key: RSA3072, fingerprint
`C5F735E977D4D45C1663AA40E4CE56B6CEF87373`, identity *restverify release
signing*; the armored public key ships at
[`docs/release-key.asc`](https://github.com/niffyhunt/restverify/blob/main/docs/release-key.asc). Recorded sha256 digests for
the current release and its validation transcript are in
[`docs/RELEASE-VALIDATION-0.3.0.md`](https://github.com/niffyhunt/restverify/blob/main/docs/RELEASE-VALIDATION-0.3.0.md); the
0.2.1 upgrade drill (exit-code matrix, artefact digests) remains at
[`docs/RELEASE-VALIDATION-0.2.1.md`](https://github.com/niffyhunt/restverify/blob/main/docs/RELEASE-VALIDATION-0.2.1.md).

## History and trends

Every verification writes one row to a local SQLite store — failures
included, because a trend that hides failures is worse than no trend.

- store: `~/.local/state/restverify/history.db` (override: `RESTVERIFY_STATE`)
- retention: newest 1000 rows per repository, pruned on write (the first
  prune announces itself on stderr)
- read it with `restverify report`, or machine-read it with
  `restverify report --json`
- browse it in a browser with `restverify dashboard` (localhost, read-only)
- `--dry-run` writes nothing

Moving the whole layout (config + state) for CI or drills:
`RESTVERIFY_HOME=/some/home`. Explicit `RESTVERIFY_CONFIG` / `RESTVERIFY_STATE`
still win.

## Security posture

- restic is invoked **read-only only** — the verbs that mutate a repository
  (backup, forget, prune…) are refused at the call site, by construction, and
  a test pins it.
- One module is permitted to spawn processes; a test pins that too.
- Zero runtime dependencies: the supply chain is Python's standard library
  and your restic binary.
- Restores go to `0700` private temp dirs; nothing persists; source paths are
  never written.
- Details and their rationale: [`docs/SECURITY.md`](https://github.com/niffyhunt/restverify/blob/main/docs/SECURITY.md).

## Platform notes

- **Linux** — fully drilled: real restic 0.16.4, real Docker for `--sandbox`,
  SFTP backend, unprivileged user.
- **Windows** — the full test suite and the run pipeline work natively
  (real `restic.exe`, Windows-cut release artefacts). `--sandbox` is the one
  Linux-only feature.
- restic: read-only verbs only, tested against 0.16.4; any recent restic
  should behave identically. If yours doesn't, that's a bug — please open an
  issue with the transcript.

## What restverify is not

- **Not a backup tool.** It never calls `backup`, `forget`, or `prune`; it
  can't lose your data because it never deletes anything.
- **Not an orchestrator.** It doesn't schedule your backups, manage
  retention, or replace resticprofile / your systemd units / your scripts.
  It sits underneath all of them and answers the question they don't: *can
  this backup actually be restored?*
- **No cloud, no dashboards, no alerting service.** Exit codes and JSON are
  the whole integration surface — anything that can run a command can consume
  a verdict.

## Documentation

| Doc | Contents |
|-----|----------|
| [`CHANGELOG.md`](https://github.com/niffyhunt/restverify/blob/main/CHANGELOG.md) | every release, honestly annotated |
| [`docs/SECURITY.md`](https://github.com/niffyhunt/restverify/blob/main/docs/SECURITY.md) | security posture and its rationale |
| [`docs/RELEASE.md`](https://github.com/niffyhunt/restverify/blob/main/docs/RELEASE.md) | how releases are cut and signed |
| [`docs/RELEASE-VALIDATION-0.2.1.md`](https://github.com/niffyhunt/restverify/blob/main/docs/RELEASE-VALIDATION-0.2.1.md) | upgrade drill, exit-code matrix, release digests |
| [`docs/I9-DRILL.md`](https://github.com/niffyhunt/restverify/blob/main/docs/I9-DRILL.md) | the 1 GiB / 1253-file scale drill |
| [`docs/I10-CI.md`](https://github.com/niffyhunt/restverify/blob/main/docs/I10-CI.md) | the CI drill, rehearsed |
| [`docs/I11-SANDBOX.md`](https://github.com/niffyhunt/restverify/blob/main/docs/I11-SANDBOX.md) | the sandbox gate |

## Status

v0.3.0 is the current release — live on
[PyPI](https://pypi.org/project/restverify/). It adds `prove` (source-less
snapshot verification with a cryptographic seal check), `dashboard` (the
local read-only history page), and `run --report-webhook` (deliver the
verdict to a collector). Every command speaks `--json` with the same
schema-1 envelope.
