Metadata-Version: 2.4
Name: bubtrsnap
Version: 0.4.0
Summary: btrfs snapshot and backup tool (single-file, stdlib-only)
Author: Gerry
License: MIT License
        
        Copyright (c) 2026 Gerry
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: homepage, https://github.com/lama7/bubtrsnap
Keywords: btrfs,backup,snapshot,send,receive,rsync
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: System :: Filesystems
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# bubtrsnap

A btrfs snapshot and backup management tool built on btrfs-native incremental
`send`/`receive`.  It bootstraps from nothing, uses a borg-style keep policy
for retention, and can send backups to local paths, remote SSH hosts, or both
in a single run.  When you need resilience across unreliable transfers, use
`rsync` as a transport to handle partial-transfer recovery.

Key features:

- Incremental snapshots and backups using `btrfs send`/`receive`
- Borg-style keep policy: hourly, daily, weekly, monthly, yearly
- Local backups (`backup_dir`), SSH remote backups (`remote_host`/`remote_path`),
  or both destinations in one run
- Optional `rsync` transport with `--rsync-opts` for partial-transfer recovery
  on interrupted sends
- Stream files (`export_file`/`export_dir`, `import_file`/`import_dir`) for
  large subvolumes or staged workflows
- Pre-snapshot, post-snapshot, and post-backup hooks with substitution strings
- Has a `--dry-run` mode that validates remote hosts and subvolumes without writing
- Minimal dependencies: `btrfs-progs` and `python3`; no databases or state files

Like the original [btrbu][] project it rewrites, bubtrsnap remains functionally
simple — give it a subvolume to snapshot and a destination to back it up to, and
it does the rest.

[btrbu]: https://github.com/lama7/btrbu

## Quick start

```bash
bubtrsnap --snapshot-dir=/pool/snapshots --backup-dir=/backup \
    archive1=/path/to/subvolume
```

Assuming a clean start, this creates a read-only snapshot in `/pool/snapshots`
and sends it to the backup location.  On subsequent runs, bubtrsnap handles
finding parents for all additional archives so only the incremental
information is sent. All paths **must** be valid btrfs subvolumes.

More than one archive can be specified:

```bash
bubtrsnap --snapshot-dir=/pool/snapshots --backup-dir=/backup \
    archive1=/var/lib data2=/home/user
```

Each archive is processed fully (pre-hook -> snapshot -> post-snapshot-hook ->
backup/send -> post-backup-hook -> keep policy) before the next begins.

## Configuration files

For anything beyond the simplest cases, a TOML configuration file keeps the
command line manageable.  The default location is
`~/.config/bubtrsnap.toml`:

```bash
bubtrsnap                     # loads ~/.config/bubtrsnap.toml
bubtrsnap --config=/path/to/configfile.toml # or use a custom file
```

The configuration file format is TOML:

```toml
# global options
snapshot_dir = "/pool/snapshots"
backup_dir = "/backup"
keep_daily = 5

[archive1]
subvolume = "/var/lib"

[archive2]
subvolume = "/home/user"
```

CLI options map to configuration keys by replacing `-` with `_` (so
`--snapshot-dir` becomes `snapshot_dir`).  Precedence is **CLI > archive >
global** — the CLI always wins, then archive-specific settings, then global.

```bash
bubtrsnap --keep-daily 3            # override keep policy for one run only
bubtrsnap archive1                # process only archive1
bubtrsnap archive2 archive3       # process multiple archives setup in a
                                  # configuration file
```

## Local and remote backups

Backups to the local system, a DAS for example, or a remote backup host
on the network can be coordinated as one, the other or both in a single
run with bubtrsnap.

### Local backup only

For a local only backup, specify `--backup-dir`:

```bash
bubtrsnap --snapshot-dir=/pool/snapshots --backup-dir=/backup \
    archive1=/path/to/subvolume
```

This takes a snapshot of the archive's subvolume and uses it as a source for
sending it to the backup location.

### Snapshots only (`--snaps-only`)

If you only want snapshots and no backups, use `--snaps-only` (config:
`snaps_only`, also per-archive):

```bash
bubtrsnap --snapshot-dir=/pool/snapshots --snaps-only \
    archive1=/path/to/subvolume
```

This creates the read-only snapshot and exits — no `btrfs send`/`receive` or
keep pruning is performed.  The `--snaps-only` option is mutually exclusive
with `--export-file`, `--import-file`, `--stage-file`, `--export-dir`,
`--import-dir`, and `--stage-dir`, and bubtrsnap will error out if
`--snaps-only` is set with any of them.

### Local sudo (`--local-sudo`)

When bubtrsnap needs to run `btrfs` commands on directories it doesn't own,
use `--local-sudo` (config: `local_sudo`).  This prepends `sudo -n` to local
`btrfs` commands (snapshot creation, `btrfs send`/`receive`, `btrfs subvolume
show` for validation) as well as `scp`/`rsync` used for file-based stream
transfers.  The user running bubtrsnap must have their sudo profile configured
for NOPASSWD access to these commands.  The `remote_sudo` option is separate —
it only affects commands run on the remote via SSH.

### Remote backup via SSH

Use the `--remote-host` and `--remote-path` options to backup to another host
over a network:

```bash
bubtrsnap --snapshot-dir=/pool/snapshots \
    --remote-host user@backuphost --remote-path /btrfs/backups \
    archive1=/path/to/subvolume
```

This creates a snapshot, then sends the appropriate incremental or full send
data via SSH to the remote host into a `btrfs receive` which writes the
resulting subvolume to `/btrfs/backups`.  The remote directory is validated as
a btrfs subvolume automatically before receiving.

The `remote_path` option always requires `remote_host`.  If `remote_sudo` is
set, `sudo -n` is prepended to the remote btrfs commands.

### Both local and remote in one run

Specify `backup_dir`, `remote_host`, and `remote_path` together and bubtrsnap
sends to both destinations using the same snapshot as source:

```bash
bubtrsnap --snapshot-dir=/pool/snapshots --backup-dir=/local/backups \
    --remote-host user@backuphost --remote-path /btrfs/backups \
    archive1=/path/to/subvolume
```

### rsync transport (optional)

By default bubtrsnap uses `scp` for file-based stream transfers to a remote.
Enable `rsync` for partial-transfer recovery:

```bash
bubtrsnap --snapshot-dir=/pool/snapshots \
    --export-file /tmp/stream.btrfs \
    --remote-host user@backuphost --remote-path /btrfs/backups \
    --rsync --rsync-opts '-e "ssh -p 2222"' \
    archive1=/path/to/subvolume
```

When `--rsync` is enabled:

- The rsync options `-a` (archive mode) and `--partial-dir .bubtrsnap-partial`
    are always added
- Additional options come from `--rsync-opts` (config: `rsync_opts`)
- Dangerous options (`--delete`, `--partial-dir`, `-v`, `--progress`,
  `--dry-run`, `--daemon`, `--rsh`, `--exclude`, etc.) are blocked and reported
- The rsync option can be set globally, per-archive, or on the CLI

To use a non-standard SSH port or identity file, put `ssh -p PORT` (or
`-i /path/to/key`) with the `-e` rsync option in `rsync_opts`:

```toml
rsync = true
rsync_opts = '-e "ssh -p 2222 -i /home/user/.ssh/id_rsa"'
```

Using rsync affects only file-based transfers (replacing scp) and does not
affect piping `btrfs send | btrfs receive` or local backup operations.

## Stream files

For very large subvolumes or when you want to break the send and receive steps
apart, use stream files:

| Option         | Description                                  |
|----------------|----------------------------------------------|
| `export_file`  | Write the btrfs stream to a file, then stop  |
| `import_file`  | Receive from a stream file into `backup_dir` |
| `export_dir`   | Write stream to `dir/archive.timestamp.btrfs`|
| `import_dir`   | Search `dir/` for the most recent matching stream file |

Local staging can be accomplished by using `export_file` and `import_file`
together: send to a file, then receive from that same file.  If the export file
already exists, it is overwritten. When combined with
`remote_host`/`remote_path`, the stream file is sent to the remote and received
there — a fully automated single-command workflow.

When `import_file` is used on its own (no snapshotting needed), the snapshot
step is skipped entirely — processing starts at the receive step.

### Remote receive with stream files

When `export_file` or `export_dir` is combined with remote settings (and no
explicit `import_file`/`import_dir`/`stage_file`), bubtrsnap:

1. Creates the snapshot
2. Writes the btrfs stream to a local file
3. Copies the stream file to the remote via `scp`
4. Runs `btrfs receive -f <temp_file> <remote_path>` on the remote
5. Cleans up the temporary file on the remote
6. Applies keep policy on the remote

For `rsync` transport, the stream file is transferred via `rsync` instead of
`scp`, and interrupted transfers can be resumed.

If `backup_dir` is configured alongside this setup, an **additional** local
piped `btrfs send | btrfs receive` writes directly to `backup_dir`. This uses
parents matched to `backup_dir`'s history — independent from the parents used
for the exported stream going to the remote. See [Combining local and remote
stream receive](#combining-local-and-remote-stream-receive) below.

### Combining local and remote stream receive

An archive configured with `backup_dir` + `export_file`/`export_dir` +
`remote_host`/`remote_path` sends to **both** destinations:

1. Local piped `btrfs send | btrfs receive` to `backup_dir`
2. Stream file written, transferred, and received on remote

Keep policy is applied independently to each destination.

Each destination receives its data through an **independent pipeline**:
`backup_dir` gets a fresh piped send from the new snapshot with parents
matched to `backup_dir`'s actual history, while the exported stream file
goes to the remote with parents matched to the remote's history. This
ensures correct incrementals even when the two backup locations have
divergent snapshots.

A CLI usage example:

```bash
bubtrsnap --snapshot-dir /snapshots \
    --export-file /tmp/stream.btrfs \
    --remote-host user@backuphost --remote-path /btrfs/backups \
    archive1=/path/to/subvol
```

This command would create a readonly snapshot of archive1, create stream file
/tmp/stream.btrfs with appropriate incremental (or not) information, use scp
to transfer the file to the remote host where it will then execute a btrfs
receive command to create the subvolume on the remote host.

In a config file:

```toml
snapshot_dir = "/snapshots"
export_dir = "/local/streams"
backup_dir = "/pool/backups"
remote_host = "user@backuphost"
remote_path = "/btrfs/backups"

[archive1]
subvolume = "/home/user/data"
keep_daily = 7
keep_weekly = 4
```

Given the above in a configuration file, a simple `bubtrsnap` CLI command
would perform the equivalent operations to the above CLI example, plus perform
a local piped send -> receive to /pool/backups and apply the specified keeps to
the archive snapshots, local backup and remote backup locations.

## Staging: stage-file / stage-dir

The `--stage-file FILE` and `--stage-dir DIR` options are a convenience that
combine send-to and receive-from using the same path, then delete the stream
file:

```bash
bubtrsnap --stage-file /tmp/archive.btrfs archive1=/path/to/subvolume
```

When combined with `--remote-host`/`--remote-path`, staging also transfers the
stream to the remote (via `scp` or `rsync`) and receives it there, just like
`export_file`/`import_file`. If `backup_dir` is set alongside remote staging,
bubtrsnap uses an independent piped send for the local backup (with correct
parents from `backup_dir`'s history) and sends the stream file exclusively to
the remote — two separate pipelines with matching parents for each destination.

## Keep policy

The keep policy is inspired by [borgbackup][].  It specifies how many
snapshots/backups to retain at each time interval:

| Option        | CLI flag         | Config key      |
|---------------|------------------|-----------------|
| `keep_hourly` | `--keep-hourly`  | `keep_hourly`   |
| `keep_daily`  | `--keep-daily`   | `keep_daily`    |
| `keep_weekly` | `--keep-weekly`  | `keep_weekly`   |
| `keep_monthly`| `--keep-monthly` | `keep_monthly`  |
| `keep_yearly` | `--keep-yearly`  | `keep_yearly`   |

Intervals are applied shortest to longest with no overlap — a snapshot kept
because of a daily keep does not count towards a weekly or monthly keep.  The
keep policy applies to **both** snapshots and backups.

When no keep values are set (all `0`), pruning is skipped — existing snapshots
and backups are left untouched.  By default, all keep values are set to 0.

```toml
snapshot_dir = "/pool/snapshots"
backup_dir = "/backup"
keep_daily = 4
keep_weekly = 2
keep_monthly = 1

[archive1]
subvolume = "/path/to/subvolume1"
```

### Forced keeps

If there is a particular archive you don't want bubtrsnap to remove, the
`--forced-keep TIMESTAMP` (config: `forced_keep`) option forces retention of
specific timestamps that would otherwise be pruned.  Format: `YYYYMMDDhhmm` (12
digits).

```bash
bubtrsnap --forced-keep 202601011200 --forced-keep 202601021200 archive1=/path/to/subvolume
```

In config:
```toml
[archive1]
subvolume = "/path/to/subvol"
forced_keep = "202601011200,202601021200"
```

Note that on the CLI, a forced keep doesn't last beyond the command itself.  A
subsequent CLI command without the forced keep specified again might result in
the archive getting pruned.  To keep a specific archive over many iterations, use `forced_keep` in a configuration file so it is specified on every run.

### Weekly boundary

Weekly keeps are aligned to the day before `week_startday` (default:
`sunday`, meaning the week ends on Saturday).  This can be set globally or
per-archive.

## Hooks

Hooks coordinate external programs at three stages:

| Hook               | Timing                           |
|--------------------|----------------------------------|
| `pre_snapshot_hook`| Before the snapshot is taken     |
| `post_snapshot_hook`| After the snapshot, before backup |
| `post_backup_hook` | After the backup is complete     |

Hooks use substitution strings that are replaced at run time:

| Placeholder   | Available in              |
|---------------|---------------------------|
| `{archive}`   | all                       |
| `{subvol}`    | pre-snapshot              |
| `{timestamp}` | all                       |
| `{snapshotdir}`| all                       |
| `{backupdir}` | all (may be empty)        |
| `{snapshot}`  | post-snapshot, post-backup |
| `{backup}`    | post-backup               |

Example:

```toml
snapshot_dir = "/pool/snapshots"
backup_dir = "/backups"

[archive1]
subvolume = "/home/user1"
pre_snapshot_hook = echo "Starting backup {timestamp}" >> ~/backuplog
post_snapshot_hook = "borg create --verbose user@server:repo::{archive} {snapshot}"
post_backup_hook = 'echo "Backup of {archive} complete" >> ~/backuplog'
```

## Dry-run and debugging

To test a particular configuration or set of options, the `--dry-run` option
shows what bubtrsnap would do without making changes:

- Read-only validation (SSH checks, `btrfs subvolume show`/`list`, stream
  header inspection) is still executed
- Write operations (snapshot creation, send/receive, delete/prune) are
  logged with a `[dry-run]` prefix and skipped
- Stream file creation and deletion are also skipped

Additionally, `--dry-run` raises verbosity to level 2 (commands) so all the
commands that would execute are visible.  Use `--debug` (equivalent to
`--verbose 3`) for maximum verbosity including UUIDs and internal decision
details.

Always use `--dry-run` to preview a new keep policy or remote configuration.

## Maintenance subcommands

Beyond the legacy `bubtrsnap ARCHIVE[=SUBVOLUME] ...` invocation, bubtrsnap
offers maintenance subcommands. When you give a subcommand name on the command
line it is parsed with its own option set; with no subcommand the legacy backup
behaviour applies (all positionals are archive names). The global options
(`--config`, `--snapshot-dir`, `--backup-dir`, `-v/--verbose`, `--dry-run`,
`--rsync`, and the `keep_*` policies) are shared by every subcommand. Run
`bubtrsnap <subcommand> --help` for that command's full option list.

**`bubtrsnap backup`** — take snapshots and send backups. This is the default
behaviour when no subcommand is given.

**`bubtrsnap list [ARCHIVE ...]`** — read-only table of the snapshot / backup /
remote timestamps that exist for each archive. All config archives are shown if
none are named.

**`bubtrsnap prune --ts TS [ARCHIVE ...]`** — delete snapshot/backup subvolumes
matching the given timestamp(s). `--ts` takes a single timestamp
`YYYYMMDDhhmm` or a dotted inclusive range `<from>..<to>`; multiple `--ts`
options match any of them. Prune never operates on "all" archives — name at
least one. Without `--yes` (and without `--dry-run`) it asks for confirmation
before deleting. If any timestamp in the target set is marked `forced_keep`,
the preview labels it `[forced-keep]` and an *additional* confirmation is
required even when `--yes` is set (a hand-picked range can capture a
protected timestamp by accident); declining aborts the whole prune.

**`bubtrsnap rebuild [ARCHIVE ...]`** — reconcile snapshot/backup/remote
locations. It restores a lost local snapshot from a local backup, restores a
lost local backup from a local snapshot (via `btrfs send`/`receive`), and
re-sends to a remote when it is missing there; it can also recover a snapshot
that exists only on the remote. It never deletes — it only restores. All
config archives are rebuilt if none are named. A rebuild is plan-first: its
action list is computed once from the state at the start of the run, and each
action fills only one of the three locations. As a result, a snapshot missing
in *two* locations needs two runs. For example, an archive that only exists as
a local snapshot gets its local backup restored on the first run and is only
re-sent to the remote on the second; likewise an archive that only exists on the
remote is recovered as a local snapshot on the first run and gets its local
backup filled in on the second. Run `bubtrsnap list` to confirm all three
locations line up before calling it done.

Examples:

```bash
bubtrsnap --config=~/.config/bubtrsnap.toml
bubtrsnap backup --snapshot-dir=/snap --backup-dir=/bck home=/home
bubtrsnap list home
bubtrsnap prune --ts 202601010000..202601050000 home --yes
bubtrsnap rebuild home
```

## Configuration reference

**Global-only options:**
`snapshot_dir`, `local_sudo`, `verbose`, `dry_run`

**Global or per-archive (archive overrides global):**
`backup_dir`, `snaps_only`, `export_dir`, `import_dir`, `stage_dir`,
`remote_host`, `remote_path`, `remote_sudo`, `rsync`, `rsync_opts`,
`week_startday`, `keep_hourly`, `keep_daily`, `keep_weekly`,
`keep_monthly`, `keep_yearly`

**Per-archive only:**
`subvolume` (required), `export_file`, `import_file`, `stage_file`,
`pre_snapshot_hook`, `post_snapshot_hook`, `post_backup_hook`, `forced_keep`

### Keep-policy precedence (archive-specific keeps)

If an archive defines **any** `keep_*` value, it fully controls its own keep
policy — the other intervals it does **not** set default to `0` (they are not
inherited from the global section). The global keep values are used only when
the archive sets none of them. So an archive-specific policy is complete as
written; to keep a yearly as well you must set `keep_yearly` too.

## Configuration examples

A set of ready-to-use configuration files ship in `examples/`. Copy one into
`~/.config/bubtrsnap.toml` (or pass it to `--config`) and edit the paths.

- `examples/basic.toml` — snapshots and local backups for **two archives** with
  a single **global** keep policy, plus `local_sudo` (with a comment explaining
  when it's needed).
- `examples/single-remote.toml` — **one archive** that snapshots, keeps a
  **local** backup, and also keeps a **remote** backup; uses an
  **archive-specific** keep policy.
- `examples/single-rsync.toml` — **one archive** that snapshots, keeps a local
  backup, and sends the **stream to the remote over rsync** (resumable
  transfers); uses an **archive-specific** keep policy.
- `examples/remote-only.toml` — **one archive** that keeps only local
  snapshots and sends backups to a **remote** (no local backup destination);
  uses an **archive-specific** keep policy.

Example — basic local snapshots and backups, two archives:

```toml
snapshot_dir = "/pool/snapshots"
backup_dir   = "/backup"

# Only needed when bubtrsnap doesn't own the subvolumes/directories (e.g.
# backing up system subvolumes or a root-owned /pool). Prepends `sudo -n`.
local_sudo = true

# Global keep policy: applies to snapshots and backups of every archive that
# does not define its own keep values.
keep_daily   = 7
keep_weekly  = 4
keep_monthly = 2

[home]
subvolume = "/home"

[varlib]
subvolume = "/var/lib"
```

Example — one archive, local + remote, archive-specific keep policy:

```toml
snapshot_dir = "/pool/snapshots"
backup_dir   = "/backup"
remote_host  = "backup@example.com"
remote_path  = "/btrfs/backup"

[server]
subvolume = "/var/lib"
# Archive-specific: fully controls this archive's policy. Intervals omitted
# here are 0 (not inherited from any global section).
keep_daily   = 14
keep_weekly  = 4
keep_monthly = 4
```

Example — one archive with rsync remote transfer (resumable stream files):

```toml
snapshot_dir = "/pool/snapshots"
backup_dir   = "/backup"
remote_host  = "backup@example.com"
remote_path  = "/btrfs/backup"

# Stream directory: incremental streams are written here as
# <archive>.<timestamp>.btrfs and rsync'd to the remote (resumable).
export_dir = "/pool/streams"

rsync      = true
rsync_opts = '-e "ssh -p 2222"'   # optional: non-standard port / key

[server]
subvolume = "/var/lib"
keep_daily   = 14
keep_weekly  = 4
keep_monthly = 4
```

**Note:** rsync only affects the *file-based* stream transfer. A plain
`btrfs send | btrfs receive` over SSH is never rsync'd, so a rsync config must
also produce a stream file (`export_file`/`export_dir`/`stage_file`).

## Installing

```bash
pipx install bubtrsnap     # recommended: puts `bubtrsnap` on your PATH, isolated from system python
pip install bubtrsnap      # or into a virtualenv (see note below)
```

**Note for Arch / other "externally-managed" distros:** on Arch (and similar distros that ship the PEP 668 "externally managed" marker), `pip install` / `pip3 install` fails with `externally-managed-environment` because it would write into distro-managed site-packages. The fix is to install into an isolated environment instead:

```bash
pipx install bubtrsnap             # simplest; pipx creates its own venv
# or
python3 -m venv ~/venvs/bubtrsnap && ~/venvs/bubtrsnap/bin/pip install bubtrsnap
```

On Arch, `pipx` itself is the `python-pipx` package: `pacman -S python-pipx` (not `pipx` or `pip3`). Avoid `--break-system-packages` unless you specifically want to override the distro guard.

## Dependencies

- `btrfs-progs` (for `btrfs` commands)
- `python3`
- SSH client (only for remote backup, at least on the client side)
- `rsync` (only when `--rsync` is enabled)

## Unit tests

```bash
python3 unittests/test_keep_policy.py -v
python3 unittests/test_ssh.py -v
python3 unittests/test_file_options.py -v
python3 unittests/test_runtime.py -v
python3 unittests/test_rebuild.py -v
```

[btrbu]: https://github.com/lama7/btrbu
[borgbackup]: https://borgbackup.org
