Metadata-Version: 2.4
Name: file-watchman
Version: 0.1.0
Summary: Work directory manifest and delta library
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Description-Content-Type: text/markdown

# file-watchman

A Python library that scans work directories, produces JSON manifests, and computes file-level deltas for incremental uploads and workflow monitoring.

## Features

- **Directory scanning** — walk a directory tree with configurable include/exclude glob rules
- **Manifest snapshots** — deterministic, sorted JSON manifests with file metadata
- **Delta computation** — detect new, changed, and deleted files between snapshots
- **Category-driven hashing** — SHA-256 for critical files (checkpoints), skip for large files (trajectories)
- **Atomic writes** — safe manifest persistence via tmp + fsync + rename
- **Zero runtime dependencies** — stdlib only, Python 3.10+

## Installation

From GitHub:

```bash
pip install git+https://github.com/ATTMOS/file-watchman.git
```

For development:

```bash
git clone https://github.com/ATTMOS/file-watchman.git
cd file-watchman
pip install -e ".[dev]"
```

## Quick Start

```python
from file_watchman import (
    ScanRules,
    HashPolicy,
    scan_manifest,
    diff_manifests,
    load_manifest,
    write_manifest_atomic,
)

# Configure scan rules
rules = ScanRules(
    includes=["**/*"],
    excludes=[".git/**", "**/__pycache__/**", "**/*.tmp"],
    max_files=50000,
)

# Configure hashing policy
hash_policy = HashPolicy(
    required_categories={"checkpoint", "restart"},
    optional_categories={"log"},
    never_categories={"trajectory"},
    max_hash_bytes=512 * 1024 * 1024,
)

# Scan a directory
manifest_new = scan_manifest(
    root="/scratch/job_123",
    rules=rules,
    hash_policy=hash_policy,
    job_id="job-123",
)

# Compare with a previous manifest
manifest_prev = load_manifest("/scratch/job_123/manifest_prev.json")
delta = diff_manifests(manifest_prev, manifest_new)

print(f"Uploads: {len(delta.uploads)}, Deletions: {len(delta.deletions)}")

# Persist the new manifest atomically
write_manifest_atomic("/scratch/job_123/manifest_new.json", manifest_new)
```

## Custom Categorization

Assign categories to files using glob-based rules:

```python
from file_watchman.rules import make_categorizer

categorize = make_categorizer([
    ("**/cpt_*.pkl", "checkpoint"),
    ("**/*.rst7", "restart"),
    ("**/*.log", "log"),
    ("**/*.nc", "trajectory"),
])

manifest = scan_manifest(
    root="/scratch/job_123",
    rules=rules,
    hash_policy=hash_policy,
    categorize=categorize,
)
```

Files not matching any rule default to the `"other"` category.

## API Reference

See [docs/api.md](docs/api.md) for the complete API reference.

## License

MIT
