Metadata-Version: 2.4
Name: lensgrep
Version: 0.1.0
Summary: Grep for your photo library — find any person across thousands of photos, locally and privately. Built-in LoRA dataset builder.
Project-URL: Homepage, https://github.com/Kris-O/lensgrep
Project-URL: Repository, https://github.com/Kris-O/lensgrep
Project-URL: Issues, https://github.com/Kris-O/lensgrep/issues
Project-URL: Changelog, https://github.com/Kris-O/lensgrep/blob/main/CHANGELOG.md
Author-email: Krzysiek Ołdak <krzysiek.oldak@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: arcface,cli,face-recognition,flux,insightface,lora,photo-library,privacy,stable-diffusion
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: End Users/Desktop
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Graphics
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Classifier: Topic :: Utilities
Requires-Python: >=3.10
Requires-Dist: faiss-cpu>=1.8.0
Requires-Dist: imagehash>=4.3.1
Requires-Dist: insightface>=0.7.3
Requires-Dist: numpy>=1.26.0
Requires-Dist: onnxruntime>=1.17.0
Requires-Dist: opencv-python>=4.9.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: platformdirs>=4.2.0
Requires-Dist: rich>=13.7.0
Requires-Dist: scikit-learn>=1.4.0
Requires-Dist: tqdm>=4.66.0
Requires-Dist: typer>=0.12.0
Provides-Extra: caption
Requires-Dist: torch>=2.2.0; extra == 'caption'
Requires-Dist: transformers>=4.40.0; extra == 'caption'
Provides-Extra: dev
Requires-Dist: mypy>=1.10.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.5.0; extra == 'dev'
Provides-Extra: gpu
Requires-Dist: onnxruntime-gpu>=1.17.0; extra == 'gpu'
Description-Content-Type: text/markdown

# lensgrep

> **`grep` for your photo library.** Find people. Remove duplicates. Build LoRA datasets. All locally, all private, no cloud.

[![PyPI](https://img.shields.io/pypi/v/lensgrep.svg)](https://pypi.org/project/lensgrep/)
[![Python](https://img.shields.io/pypi/pyversions/lensgrep.svg)](https://pypi.org/project/lensgrep/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![CI](https://github.com/Kris-O/lensgrep/actions/workflows/ci.yml/badge.svg)](https://github.com/Kris-O/lensgrep/actions/workflows/ci.yml)

```
$ lensgrep scan ~/Photos
scanned 12,847 files  faces+= 38,219  clustered 412 groups

$ lensgrep name 3 "Mom"
✓ 'Mom' ← cluster 3, assigned 1,204 face(s)

$ lensgrep find Mom -n 5 --paths
/home/me/Photos/2023/birthday/IMG_0341.jpg
/home/me/Photos/2024/cabin/DSC_8821.jpg
/home/me/Photos/2024/cabin/DSC_8824.jpg
/home/me/Photos/2025/holidays/IMG_1129.jpg
/home/me/Photos/2025/holidays/IMG_1131.jpg

$ lensgrep lora Mom ./mom_dataset --size 50
✓ LoRA dataset: 50 images at 1024×1024 → ./mom_dataset
  Token: mom
  Next: open AI Toolkit / kohya_ss → point dataset path here → train.

$ lensgrep dupes --near --threshold 5
Found 412 group(s), 1,847 extra file(s), could free 14,203.5 MB.
…
Read-only listing. Re-run with --mode delete or --mode symlink to act.
```

## What it is

**lensgrep** is a single-purpose tool that does one thing well: **find photos of a specific person in a folder of photos**, no matter how big the folder is. SQLite under the hood, [InsightFace](https://github.com/deepinsight/insightface) ArcFace embeddings, FAISS for vector search. Nothing leaves your machine.

It's intentionally small. If you want a full self-hosted Google Photos clone, run [PhotoPrism](https://github.com/photoprism/photoprism) or [Immich](https://github.com/immich-app/immich). If you want `grep` for faces, run this.

## Killer features

### Duplicate detection — reclaim disk space

Most photo libraries have 10–30% wasted space: phone-sync dupes, edited copies, burst shots, JPEG-vs-RAW pairs, "the same file in three folders because cloud sync went sideways."

```bash
lensgrep dupes                          # exact (bit-identical) only — read-only listing
lensgrep dupes --near --threshold 5     # near-duplicates: resized / re-saved / minor edits
lensgrep dupes --near --mode delete     # actually delete the victims
lensgrep dupes --near --mode symlink    # replace victims with symlinks to the keeper
lensgrep dupes --keep newest            # alternative: oldest | newest | shortest-path | largest (default)
```

**Two detection modes:**
- **Exact** — SHA-256 hash of the file, zero false positives, catches bit-identical copies regardless of filename.
- **Near** — perceptual hash (pHash) with configurable Hamming-distance threshold. Catches resized copies, JPEG re-saves, minor color edits, thumbnails — the stuff that exact match misses.

**Safety by default:** without `--mode delete` or `--mode symlink`, the command is read-only. You'll always see what it *would* do before it does anything.

### LoRA dataset builder — flagship

Training a face LoRA (Flux / SDXL) usually means manually picking 40–60 diverse photos of one person from your library. lensgrep does this in one command:

```bash
lensgrep lora "Krzysiek" ./dataset --size 50 --resolution 1024
```

This:
1. Pulls all photos containing Krzysiek from the indexed library
2. Scores quality (sharpness, face area, detection confidence)
3. Picks 50 **diverse** photos via greedy farthest-point sampling on ArcFace embeddings — no near-duplicates, balanced mix of closeup / portrait / half-body / full-body
4. Crops to 1024×1024 with the face in the upper third (rule-of-thirds)
5. Writes `.txt` caption files next to each image, ready for [AI Toolkit](https://github.com/ostris/ai-toolkit) or [kohya_ss](https://github.com/bmaltais/kohya_ss)

You go from "8 000 raw photos" to "trainable Flux LoRA dataset" in **one command**.

## Install

```bash
pip install lensgrep
```

Optional: GPU support (10× faster scan)

```bash
pip install "lensgrep[gpu]"
```

Requires Python 3.10+. First run downloads ~280 MB of InsightFace models to `~/.insightface/`.

## Quickstart

```bash
# 1. scan your photo library (one time, ~1.5h CPU / ~10 min GPU for 10k photos)
lensgrep scan ~/Photos

# 2. browse detected face clusters
lensgrep people --min-size 10

# 3. name the ones you care about
lensgrep name 3 "Mom"
lensgrep name 7 "Dad"
lensgrep name 12 "Krzysiek"

# 4. find photos of someone
lensgrep find Mom -n 100
lensgrep find Mom --paths | xargs -I{} cp {} ./mom_photos/

# 5. export all photos of a person
lensgrep export Mom ./mom_photos --mode symlink

# 6. build a LoRA training dataset
lensgrep lora Krzysiek ./krzysiek_dataset --size 50

# 7. quick stats
lensgrep stats
```

## Commands

| Command | What it does |
|---------|--------------|
| `scan <PATH>` | Recursively scan folders for faces. Incremental by default (skips unchanged files). |
| `people` | List detected face clusters, with face counts. |
| `name <CLUSTER> <NAME>` | Tag a cluster with a person's name. |
| `find <NAME>` | List photos of a tagged person, ranked by face quality. |
| `export <NAME> <DIR>` | Copy/symlink/hardlink all matching photos to a folder. |
| `lora <NAME> <DIR>` | Build a Flux/SDXL LoRA training dataset. |
| `dupes` | Find (and optionally remove) duplicate photos — exact or perceptual. |
| `recluster` | Re-run DBSCAN with custom parameters. |
| `stats` | Database statistics. |

Run `lensgrep <command> --help` for full options on any command.

## How it works

```
                 ┌────────────────┐
   ~/Photos ───▶ │  scan          │ ──▶ images table  (SQLite)
                 │  - detect      │
                 │  - embed (512d)│ ──▶ faces table   (SQLite, embedding as BLOB)
                 │  - score       │
                 └────────────────┘
                          │
                          ▼
                 ┌────────────────┐
                 │  cluster       │ ──▶ persons table (named clusters)
                 │  (DBSCAN)      │
                 └────────────────┘
                          │
                          ▼
                 ┌────────────────┐
                 │  find / export │
                 │  / lora        │
                 └────────────────┘
```

- **Detection + embedding:** InsightFace `buffalo_l` (SCRFD detector + ArcFace r50 recognition). State-of-the-art, ONNX, runs on CPU or CUDA.
- **Storage:** Plain SQLite — no Docker, no daemons, no cloud accounts. The whole index lives in one file at the platform-standard data dir.
- **Vector search:** FAISS `IndexFlatIP` on L2-normalized embeddings — exact cosine similarity, sub-millisecond on a million faces.
- **Clustering:** DBSCAN over cosine distance, `eps=0.35`, `min_samples=5`. Tune via `lensgrep recluster --eps 0.4 --min-samples 8`.
- **LoRA selection:** Greedy farthest-point sampling on embeddings — maximizes diversity. Quality scoring weights sharpness (Laplacian variance), face area, detection confidence.

## Why this exists

In 2026, the photo-management landscape has two extremes:

- **Bloated cloud apps** that ingest your library and "do AI on it": Google Photos, Apple Photos, Amazon Photos. Hand over your private life.
- **Self-hosted galleries with face features bolted on**: PhotoPrism, Immich, digiKam. Powerful but heavy — Docker, databases, web UIs, hours of setup, weeks of curating.

Neither helps if you just want to answer one question:

> "Where are all the photos of [person] in this big folder?"

…or:

> "Why does this folder weigh 80 GB? How many of these photos are actual duplicates?"

…or:

> "I'm training a Flux LoRA of my partner. Give me 50 diverse, high-quality photos."

That's the niche. **One job. Done well.**

## Comparison

|  | lensgrep | PhotoPrism | Immich | digiKam | face_recognition |
|---|---|---|---|---|---|
| CLI-first | ✅ | ❌ | ❌ | ❌ | ✅ (library) |
| No Docker / DB setup | ✅ | ❌ | ❌ | ✅ | ✅ |
| Duplicate detection (exact + perceptual) | ✅ | partial | partial | partial | ❌ |
| LoRA dataset export | ✅ | ❌ | ❌ | ❌ | ❌ |
| FAISS vector search | ✅ | ❌ | ✅ | ❌ | ❌ |
| Modern face model (ArcFace) | ✅ | ✅ | ✅ | ❌ (older) | ❌ (dlib) |
| Local-only / no cloud | ✅ | ✅ | ✅ | ✅ | ✅ |
| Install in 10 seconds | ✅ | ❌ | ❌ | ❌ | ✅ |

## Privacy

Everything stays on your machine. No telemetry, no analytics, no network calls except the one-time model download from GitHub releases on first run. The SQLite database is plain text — you can `sqlite3 lensgrep.db ".dump"` and read it.

If you want to verify: `pip install lensgrep` ships ~600 lines of pure Python source. Audit it in an afternoon.

## Performance

Rough numbers on a 2020 laptop (Quadro RTX 5000 mobile, 16 GB VRAM):

| Library size | CPU scan | GPU scan (CUDA) |
|---|---|---|
| 1 000 photos | ~3 min | ~30 s |
| 10 000 photos | ~30 min | ~5 min |
| 100 000 photos | ~5 h | ~50 min |

Scan is the only slow part. After indexing, `find` / `export` / `lora` are sub-second on hundreds of thousands of faces.

CPU works fine if you're willing to leave it running overnight once. GPU setup needs CUDA Toolkit 12 + cuDNN 9 (`pip install "lensgrep[gpu]"` is necessary but not sufficient — see [docs/gpu.md](docs/gpu.md)).

## Roadmap

- [x] CLI MVP (scan, people, name, find, export, lora, dupes, stats)
- [ ] Per-person nearest-neighbor add: `lensgrep find-similar <photo.jpg>`
- [ ] EXIF date timeline filter: `lensgrep find Mom --year 2024`
- [ ] EXIF GPS map output
- [ ] HEIC support
- [ ] `lensgrep serve` — read-only web UI (FastAPI + minimal frontend) for browsing
- [ ] BLIP / Florence-2 auto-captions for richer LoRA training data

If the project gains traction, the web UI is next. Open an issue if you want to push it forward.

## Contributing

PRs welcome. Project conventions:

- Ruff for lint + format (`ruff check . && ruff format .`)
- Pytest for tests (`pytest`)
- Conventional commits (`feat:`, `fix:`, `docs:`, `refactor:`)

```bash
git clone https://github.com/Kris-O/lensgrep
cd lensgrep
pip install -e ".[dev]"
pytest
```

## License

MIT — see [LICENSE](LICENSE).

## Acknowledgments

Standing on the shoulders of:
- [InsightFace](https://github.com/deepinsight/insightface) — face detection + recognition
- [FAISS](https://github.com/facebookresearch/faiss) — vector search
- [Typer](https://github.com/fastapi/typer) + [Rich](https://github.com/Textualize/rich) — joy of CLI dev

Inspired by the question: *why isn't this just one command yet?*
