Metadata-Version: 2.5
Name: dirark
Version: 0.4.1
Summary: Archive and compress directory trees
Project-URL: Homepage, https://github.com/endremborza/dirark
Author-email: Endre Márk Borza <endremborza@gmail.com>
License: Copyright 2026 Endre Márk Borza
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
License-File: LICENSE
Requires-Python: >=3.12
Requires-Dist: protocli
Description-Content-Type: text/markdown

# dirark

[![pypi](https://img.shields.io/pypi/v/dirark.svg)](https://pypi.org/project/dirark/)

Content-addressed directory archival with deduplication and remote sync.

Archives directories into compressed, deduplicated stores indexed by BLAKE2b checksums. Supports incremental archiving (idempotent re-runs), merging arks, and pushing/pulling to remote storage via rsync.

## Requirements

- Python >= 3.12
- System utilities: `rsync`, `tar`, `zstd`

## Install

```bash
uv add dirark
# or
pip install dirark
```

## CLI

```bash
# Archive a directory → creates <src>.ark.d
dirark archive <src_dir>

# Restore all files from an ark
dirark restore <ark_dir> <dest_dir>

# Print a single file's contents to stdout
dirark read <ark_dir> <file_path>

# Merge an ark into a remote one and back (local path or user@host:/path);
# only objects the other side lacks are transferred, nothing is overwritten
dirark push <local_ark> <remote>
dirark pull <remote> <local_ark>

# Merge src_ark into dst_ark
dirark merge <src_ark> <dst_ark>

# Archive a directory and add it to a remote ark (created if absent)
dirark add <src_dir> <remote_ark>
```

`dirark --help-all` prints every command's arguments and flags.

## Python API

```python
from dirark import archive_dir, restore_ark, ArkReader
from dirark import push_ark, pull_ark, merge_arks, add_dir_to_remote_ark
from pathlib import Path

# Archive
archive_dir(Path("my_dir"))             # creates my_dir.ark.d
archive_dir(Path("my_dir"), ark_out=Path("existing.ark.d"))  # merge into existing

# Restore
restore_ark(Path("my_dir.ark.d"), Path("restored/"))

# Read individual files without full restore
with ArkReader(Path("my_dir.ark.d")) as ark:
    files = ark.list_files()
    checksum = ark.get_checksum("path/to/file.txt")
    data = ark.read_file("path/to/file.txt")

# Remote sync: index-first merges, so arks that diverged (one per machine) can share a remote
push_ark(Path("my_dir.ark.d"), "user@host:/backups/my_dir.ark.d")
pull_ark("user@host:/backups/my_dir.ark.d", Path("local.ark.d"))

# Merge two local arks
merge_arks(Path("src.ark.d"), Path("dst.ark.d"))

# Archive locally and push to remote in one step
add_dir_to_remote_ark(Path("my_dir"), "user@host:/backups/archive.ark.d")
```

## Blob Store

`BlobStore` exposes the same compressed, deduplicated storage as a content-addressed
bytes interface — useful for caching computed artifacts keyed by their content.

```python
from dirark import BlobStore
from pathlib import Path

store = BlobStore(Path("cache"))

# write() stages bytes and returns their BLAKE2b checksum; it is idempotent
checksum = store.write(b"...payload...")
assert store.has(checksum)
assert store.read(checksum) == b"...payload..."

# write() is safe from many processes at once (atomic staging). Once writers
# have stopped, commit() packs staged blobs into the ark's tar.zst archives.
store.commit()

# Pull blobs from another store (e.g. a per-worker shard) into this one
store.merge_from(BlobStore(Path("worker-0")))

# Sync committed blobs with a store rooted at a remote path (staging stays local)
store.push("user@host:/backups/cache")
store.pull("user@host:/backups/cache")
```

Checksums are computed with the same BLAKE2b-512 hashing used by `archive_dir`, so a blob and
an archived file with identical content share a key and deduplicate against each
other — including across `merge_from`.

## Storage Format

An ark is a directory containing:

- `index.sqlite` — maps file paths to BLAKE2b checksums, and checksums to tar archives
- `data-NNNNN.tar.zst` — zstd-compressed tars holding the actual file data (batched at `MAX_TAR_MB` of source bytes each, smaller in a `BlobStore`, where every read decompresses a whole tar; a single larger file gets its own tar)

Files are deduplicated by checksum. Re-archiving a directory that hasn't changed is a no-op.

## Development

Performance benchmarks live in `benchmarks/` and exist to settle implementation
trade-offs with measurement rather than assumption:

```bash
uv run python benchmarks/bench.py            # full
uv run python benchmarks/bench.py --quick    # smaller inputs, fewer reps
```
