Metadata-Version: 2.4
Name: clearclutter
Version: 0.1.2
Summary: Non-destructive photo/video dedup: scan a drive, review duplicates in a Google-Photos-style UI, move rejects to a bin folder
Author: Manikandan Balasubramanian
License: MIT
Project-URL: Homepage, https://github.com/ManikandanBalasubramanian/ClearClutter
Keywords: photos,dedup,duplicates,phash,photo-management
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: Multimedia :: Graphics
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pillow
Requires-Dist: pillow-heif
Requires-Dist: imagehash
Requires-Dist: xxhash
Requires-Dist: numpy
Requires-Dist: fastapi
Requires-Dist: uvicorn[standard]
Provides-Extra: faces
Requires-Dist: insightface; extra == "faces"
Requires-Dist: onnxruntime; extra == "faces"
Requires-Dist: opencv-python-headless; extra == "faces"
Dynamic: license-file

# ClearClutter

Non-destructive photo/video dedup for a messy drive, with a Google-Photos-style
review UI. Scans a source tree into a SQLite index, puts **every file into a
group** (duplicates together, unique files as groups of one), and gives you a
daily verify queue. **It never deletes anything** — rejected copies move to
`<root>/ClearClutter_bin/` (folder structure preserved) for you to delete
manually, every move is logged, and the last batch is always undoable.

## Install

```bash
pip install clearclutter            # from PyPI, once published
# or from a checkout:
pip install .
# with face recognition ("People"):
pip install "clearclutter[faces]"
```

## Usage

```bash
clearclutter scan /Volumes/WD       # index the drive (incremental)
clearclutter group /Volumes/WD      # build groups (dupes + singles)
clearclutter faces /Volumes/WD      # optional: cluster faces by person
clearclutter serve /Volumes/WD      # review UI at http://127.0.0.1:8177
```

**The index lives on the drive itself** — `<root>/.clearclutter/` holds the
SQLite database, thumbnails, and the move audit log, so every disk is fully
self-contained and travels with its data. With several disks plugged in, run
one `clearclutter serve <root>` per disk; each instance picks the next free
port automatically and shows its drive name in the top bar.

## Staying in sync

- On server start, a fast background probe (walk + stat, no hashing) counts
  files that are new, changed, or missing versus the index; the **Sync**
  button shows the number.
- **Sync** rescans incrementally (only new/changed files are hashed) and
  rebuilds groups, with live progress. New files land as unverified groups —
  straight into your review queue.
- **Manually moved files are reconciled by content hash**: if you move or
  rename a verified file outside the app, the rescan matches the "missing"
  row to the new path and carries its verified flag and tags over.
- Files that are truly gone (deleted or moved off-drive) show as *broken
  links* in the top bar with a one-click **clean up** (index rows only —
  no files are ever touched).

## How grouping works

| Group type | Signal |
|---|---|
| `exact` | identical bytes (xxh3) — any name, folder, or source |
| `near` | perceptual hash within a tunable threshold — re-encodes, resizes, Google Photos re-downloads, bursts |
| `live` | Apple ContentIdentifier — Live Photo HEIC+JPEG+MOV trios (keeper: the HEIC) |
| `video_time` | videos captured within 120 s — manual review aid |
| `single` | everything else: a unique file is its own group of one |
| `manual` | groups you created by splitting others |

Names are never used for matching, so same-name-different-photo never groups.
The suggested keeper (★) is scored by format (RAW > HEIC > JPEG > PNG),
resolution, EXIF date, GPS, camera, and size — override with one click.
**Degrouping** wrongly-matched photos splits them into their own `manual`
group; never-pair rules and manual groups both survive regrouping.

## The verify workflow

Every file is **unverified** until you deal with its group:

- **Keep ★, bin rest** — queues the duplicates for the bin and marks the whole
  group verified.
- **Verify** — marks a group correct as-is (singles, or groups where you keep
  everything).
- Bulk-select photos in any timeline and hit **Verify** in the action bar.

**Groups (Unverified)** in the left nav is your daily queue; the badge shows
what's left. The **Verified** view shows everything you've already cleared.

## The UI

Left nav: **All Data · Photos · People · Location · Verified ·
Groups (Unverified) · Videos**, plus your tags and the pending-actions queue.
Timelines use a justified grid with date headers, hover-select circles, and a
bulk action bar (verify / tag / move / bin). Clicking a photo opens a lightbox
with arrow-key navigation and an info panel. **People** clusters faces
(incremental — names survive re-runs). **Location** clusters GPS on a ~1 km
grid. Light and dark theme follow the OS.

Applying moves: **Pending actions → Dry run → Apply**. Undo restores the last
batch. Tags live in SQLite only — they're a semantic view, independent of
folders.

## Built for volume

Designed around a 1 TB / few-hundred-thousand-file library: one streaming read
per file (hash computed while reading), incremental rescans skip unchanged
files by size+mtime, thumbnails are sharded across 256 subdirectories, group
statistics are precomputed at build time (the review list is a pure indexed
read — no aggregation per page), timelines use an expression index on the
capture date, singleton groups insert in bulk, and every list in the UI is
paginated with infinite scroll. Scans and syncs run in a background thread
with progress — the UI never blocks.

## Notes

- Workspace (index, thumbnails, `moves.jsonl`): `<root>/.clearclutter`
  (`--workspace` to override)
- `exiftool` on PATH enables Live Photo pairing and video dates (`brew install exiftool`)
- phash threshold 6 is conservative; raise toward 10 to catch heavier edits
- NTFS on macOS needs Paragon/Tuxera for the apply step; scanning is read-only
