Metadata-Version: 2.4
Name: navig-explore
Version: 0.2.1
Summary: NAVIG Explore module — a universal local media/data explorer for ANY folder. Buckets files by type (image/gif/video/audio/text/document/archive), auto-reads Telegram exports, previews everything, and organizes by drag-to-class. Non-destructive. A first-party navig plugin (free, toggleable).
Author-email: NAVIG Development Team <opensource@navig.run>
License-Expression: Apache-2.0
Project-URL: Homepage, https://navig.run
Project-URL: Documentation, https://navig.run/docs
Keywords: navig,cli,agent,automation,plugin,explore,files,media,telegram,browser
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Utilities
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: navig-sdk>=2.7.0
Requires-Dist: typer>=0.14.0
Requires-Dist: rich>=13.7.0
Requires-Dist: Pillow>=9
Provides-Extra: vision
Requires-Dist: torch>=2.2; extra == "vision"
Requires-Dist: open_clip_torch>=2.24; extra == "vision"
Requires-Dist: opencv-python-headless>=4.9; extra == "vision"
Requires-Dist: scikit-learn>=1.3; extra == "vision"
Requires-Dist: scipy>=1.11; extra == "vision"
Requires-Dist: imagehash>=4.3; extra == "vision"
Requires-Dist: pytesseract>=0.3.10; extra == "vision"
Requires-Dist: transformers>=4.40; extra == "vision"
Requires-Dist: numpy>=1.24; extra == "vision"
Dynamic: license-file

# navig-explore

<!-- navig:plugin-card:start -->
<!-- Generated by scripts/gen_plugin_catalog.py — edit the plugin's pyproject.toml, not this block. -->

> A universal local media/data explorer for ANY folder. Buckets files by type (image/gif/video/audio/text/document/archive), auto-reads Telegram exports, previews everything, and organizes by drag-to-class. Non-destructive.

**Works on its own** — its own CLI, no navig required · v0.2.1 · [PyPI](https://pypi.org/project/navig-explore/)

![navig-explore demo](https://raw.githubusercontent.com/navig-run/plugins/main/navig-explore/docs/demo.gif)

**On its own** (no navig needed)

```bash
pip install navig-explore
navig-explore --help   # no navig needed
```

**With navig**

```bash
pip install navig-explore
navig explore --help
```

<details><summary>Commands</summary>

| Command | What it does |
|---|---|
| `navig explore` | Explore & organize any folder in a local web UI. |
| `navig explore audio-ingest` | 🗂️ Ingest clips into a sound library — classified, renamed, cleaned to mp3, reversible. |
| `navig explore audio-ingest-undo` | ↩️  Put every ingested original back and remove the mp3s made from it. |
| `navig explore audio-sort` | 🎧 Sort audio by CONTENT — music / voice / sfx — from the waveform, not the filename. |
| `navig explore audio-undo` | ↩️  Put every file an audio-sort run moved back where it came from. |
| `navig explore collect` | 🎯 Pull a themed selection (e.g. --res 4K --year 2024) into an edit folder via hardlinks (no copy). |
| `navig explore dedup` | 🎯 Find exact + identical-pixel (auto-trash) and near (flag-only) duplicates. |
| `navig explore gallery` | 🖼️  Build a shareable offline gallery.html — cached thumbnails, live filter, lightbox. |
| `navig explore merge-episode` | 🎬 Merge an episode's clips into one lossless RAW file (MPEG-TS concat; skips corrupt clips). |
| `navig explore open` | 🔎 Browse & organize a folder in the local web UI (the default for `navig explore <folder>`). |
| `navig explore photos arrange` | 🗂️  Folders per person, year, place or kind — as links, moving nothing. |
| `navig explore photos arrange-remove` | ↩️  Delete an arranged folder tree. Originals are never affected. |
| `navig explore photos classes` | 🏷️  Re-decide what each file IS — file metadata first, guess second. |
| `navig explore photos dates` | 📅  Recover capture dates EXIF no longer holds — and record HOW. |
| `navig explore photos events` | 📅  Work out which occasion each photograph belongs to. |
| `navig explore photos explain` | 🔍  Show every fact the catalog holds about one photo, and where it came from. |
| `navig explore photos faces` | 🙂  Find and embed faces — only in files that are actually photographs. |
| `navig explore photos gallery` | 🖼️  A browsable page of the library, filtered by person, year, place and kind. |
| `navig explore photos index` | 🧠  Read every photo ONCE — hash, measure, embed and classify it. |
| `navig explore photos organize` | 🖼️  File photos into <date-root>/YYYY/YYYY-MM by EXIF, quarantine litter and salvage. |
| `navig explore photos people cluster` | Group the detected faces. Names you have already assigned are preserved. |
| `navig explore photos people list` | Show face groups, biggest first. |
| `navig explore photos people merge` | Fold groups together — clustering always over-splits the same person. |
| `navig explore photos people name` | Name a group and lock it, so re-clustering never loses the name. |
| `navig explore photos people sheet` | 🖼️  One row of faces per group, labelled with the id — so you can name them. |
| `navig explore photos people split` | Detach a group so the next cluster run re-derives it. |
| `navig explore photos places` | 🌍  Work out where photos were taken — GPS, neighbours, then folder names. |
| `navig explore photos search` | 🔎  Find photos by what is IN them — plus person, place, year and kind. |
| `navig explore photos status` | 📊  What the vision catalog currently knows about this library. |
| `navig explore photos subjects` | 🏷️  Tag what photographs are OF — sunsets, food, the sea, cars, concerts. |
| `navig explore photos triage` | 🗂️  Plan the split: real photos vs webcam vs screenshots vs web litter. |
| `navig explore photos triage-undo` | ↩️  Put every file a triage run moved back where it came from. |
| `navig explore photos undo` | ↩️  Put every file a photos-organize run moved back where it came from. |
| `navig explore photos views` | 🗂️  Build the whole organised tree: year, event, person, place, screenshots. |
| `navig explore photos views-remove` | ↩️  Delete the organised tree. Your photos are never affected. |
| `navig explore probe` | 📐 Extract width/height/codec/date/GPS/camera → .mediaexplorer/meta.jsonl (resumable). |
| `navig explore rename apply` | Move every file in the plan (MD5-checked), then rewrite references to them. |
| `navig explore rename plan` | Compute every new name and write the plan CSV. Moves NOTHING — read it first. |
| `navig explore rename undo` | Restore rewritten text files from their backups and move every file back. |
| `navig explore rename verify` | Every file where the manifest says, same MD5, nothing left behind, no old path named. |
| `navig explore route` | 🚚 Route media to the master library. Default: PLAN (manifest only). --apply to move. |
| `navig explore undo` | ↩️  Put every file a quarantine / photos-organize run moved back where it was. |
| `navig telegram-exports audit` | Chained-move-aware integrity check: prove no export was lost. |
| `navig telegram-exports dedupe` | Find loose files that are byte-identical to a copy already inside their export. |
| `navig telegram-exports match` | Pair loose downloaded files back to the source chat by original name (+ exact size). |
| `navig telegram-exports notebook build` | Copy EXPORT into OUT/_export/<date> (sha256-verified) and (re)generate the Markdown. |
| `navig telegram-exports notebook faces` | Group the faces in the People photos (locally) and draw a sheet to name them from. |
| `navig telegram-exports notebook name` | Give a face group a name; its photos are regrouped (and moved) under it. |
| `navig telegram-exports notebook ocr` | Read the text inside every image (local Tesseract) so screenshots become searchable. |
| `navig telegram-exports notebook photos` | Sort images by what they show (people, places, screenshots, documents…) with a local model. |
| `navig telegram-exports notebook place` | Move every media file out of _export/ into real folders under Media/ (logged, reversible). |
| `navig telegram-exports notebook quarantine` | Move extra byte-identical media copies from _export/ to _duplicates/ (logged, restorable). |
| `navig telegram-exports notebook restore` | Put every quarantined duplicate back where the export had it. |
| `navig telegram-exports notebook search` | Find a message or screenshot by any word in it — its text, the text in its image, |
| `navig telegram-exports notebook unplace` | Move every placed file back into _export/ exactly where the export had it. |
| `navig telegram-exports notebook verify` | Coverage gate: prove every message, link and file made it into the library. |
| `navig telegram-exports organize` | File loose `ChatExport_*` folders at ROOT into `Category/Chat/Date`. |
| `navig telegram-exports verify` | Re-classify filed exports and report any that no longer match their category. |

</details>

<!-- navig:plugin-card:end -->

NAVIG Explore module — a universal local media/data explorer for ANY folder. Buckets files by type (image/gif/video/audio/text/document/archive), auto-reads Telegram exports, and previews everything. A first-party navig plugin (free, toggleable).

**License:** Apache-2.0 · **Version:** 0.1.0

## What it is

`navig-explore` is a **first-party NAVIG plugin** — it extends NAVIG itself (CLI verbs +
gateway routes), and it is free and toggleable. Point it at any folder and it buckets the
files by type, previews them, and understands Telegram chat exports. Plugins are different
from *blocks*: a plugin adds capabilities to the platform; a block is an outcome you *apply*
with `navig apply`.

## Commands

```
navig explore <folder>              # bucket + preview any folder's files by type
navig explore audio-sort <dir>      # sort audio by CONTENT: music / voice / sfx
navig explore audio-undo <log>      # put an audio-sort run back exactly as it was
navig explore photos organize <dir> # file a photo library by EXIF capture date
navig explore photos undo <log>     # put a photos run back exactly as it was
navig telegram-exports <path>       # read a Telegram data export
```

### `photos organize` — date a photo library without wrecking it

Files photos into `<date-root>/YYYY/YYYY-MM/` from **EXIF capture dates**, quarantines
litter and undatable recovery salvage, and optionally moves video out of the photo tree.

```
navig explore probe   "X:\Photos"                    # capture dates (required)
navig explore dedup   "X:\Photos"                    # duplicate copies (optional)
navig explore photos organize "X:\Photos" \
    --video-dest "X:\Video\from-photos"              # PLAN only — review the CSV
navig explore photos organize "X:\Photos" --apply    # execute
```

`dedup` reports three classes, and only the first two are ever auto-quarantined:

| class | what it is | auto-trash |
|---|---|---|
| `exact` | byte-identical | yes |
| `identical-pixels` | **decodes to the same image**, differs only in container/metadata | yes |
| `near-image` / `near-video` | perceptually close (resize, crop, re-encode) | no — human decides |

`identical-pixels` is the class byte-hashing structurally cannot see: re-saving a JPEG or
writing one EXIF tag changes every byte while leaving every pixel untouched. On a real
family album, **184 of 210** name collisions were exactly this — SHA-256 alone would have
kept all 184 as `(1)` twins of photos already in the library. It costs no extra I/O (the
digest is taken during the decode the perceptual pass already performs), needs only a
decoder — not `imagehash` — and keeps the copy carrying the *most* metadata. Opt out with
`photos organize --no-pixel-dupes`, or skip the decode pass entirely with
`dedup --exact-only`.

**Acting on the report — `--quarantine`:**

```
navig explore dedup "X:\Photos"                 # report only
navig explore dedup "X:\Photos" --quarantine    # ...and move the redundant copies
navig explore undo  "<log printed above>"       # put every one of them back
```

Redundant copies move to `<root>/.trash/dupes/` keeping their relative path — nothing is
deleted. Only `exact` and `identical-pixels` are moved; a `near-image` cluster is often a
burst of consecutive shots rather than one photo twice, so those are always left for a
human. Overlapping groups are merged into connected components first, so a file that is
byte-identical to one copy *and* pixel-identical to a third resolves to exactly one
survivor in a single pass, and a group can never lose every copy.

What it refuses to do, and why it matters on a real archive:

- **Never flattens human-named folders.** A real date tree is not uniform — it holds
  `2009/Alzon 2009`, `2009/14 juillet 2009`, `2014/Берлин 2014`. Those names are the
  best organisation in the library and nothing can rebuild them. Only `YYYY-MM`
  buckets, `YYYY-00` stubs, `dd.mm.yy` day folders and folders you declare with
  `--loose` are ever reshaped; curated collections are read-only.
- **Never invents a date.** Capture dates come from EXIF only — never mtime. Archives
  assembled by recovery tools carry meaningless timestamps (that is how `1970` and
  `2031` folders appear). No capture date ⇒ the photo stays undated instead of being
  filed under a guess.
- **Never mistakes a caption for litter.** A zero-byte `.txt` is often the only
  description a folder has, with the filename as the note (`a la plage.txt`).
- **Never deletes.** Quarantine moves to `<root>/.trash/{junk,tiny,dupes}` keeping the
  original relative path, and every run writes an undo log *as it goes* — a run killed
  halfway is still fully reversible via `photos undo`.

Plan-by-default is the review gate: the first invocation only writes
`.mediaexplorer/photos-plan.csv`.

### `audio-sort` — sort a sound dump by what it *sounds* like

Ripped clips have useless names (`#fyp_#viral_7593….m4a`) and no tags, so bucketing by
type or filename can't tell a song from someone talking. `audio-sort` decides from the
waveform, using ffmpeg + numpy only — no model download, no GPU:

```
navig explore audio-sort "X:\Audio\TikTok\music" \
  --learn music="X:\Audio\Music\Genres" \
  --learn voice="X:\Audio\Spoken Word" \
  --learn sfx="X:\Audio\SFX\Foley Sound Effects" \
  --route music="X:\Audio\Production\Video Edit Music" \
  --route voice="X:\Audio\TikTok\voices" \
  --route sfx="X:\Audio\SFX\TikTok Sounds" \
  --min-conf 0.6 --apply
```

It trains on folders **you already sorted**, prints a cross-validated accuracy before
touching anything, and defaults to plan-only. Anything below `--min-conf` goes to
`_review/` rather than being guessed at, sidecars (`.md` transcripts) travel with their
audio, name collisions are suffixed rather than overwritten, and every move is logged so
`audio-undo` restores the folder exactly.

**Targeted cleanup.** `--min-dur` / `--max-dur` / `--keep` narrow what a run may touch,
so you can fix the misfits without reorganising the folder:

```
# pull the hour-long albums out of a sound-effects library,
# and leave the actual sound effects exactly where they are
navig explore audio-sort "X:\Audio\SFX" --model model.json \
  --route "music=X:\Audio\Music\Recovered from SFX" \
  --route "voice=X:\Audio\Spoken Word\Recovered from SFX" \
  --keep sfx --min-dur 120 --apply
```

Out-of-scope files are reported as `leave` in the plan rather than filtered out — a plan
that silently omitted them would look like it covered the folder when it deliberately
did not.

If `faster-whisper` is installed, its bundled Silero VAD adds speech-activity features —
the strongest single cue for "is someone talking?".

### Rename a whole tree to English — `rename plan | apply | verify | undo`

For a folder that grew out of Notion exports and several languages: Cyrillic folder names,
titles cut at 50 characters, `(2)` copies, emoji, `#hashtags`, Notion hashes.

```
navig explore rename plan   ROOT --map map.yaml --out plan.csv     # moves NOTHING — read the CSV
navig explore rename apply  ROOT --plan plan.csv --map map.yaml --log-dir LOG --yes
navig explore rename verify ROOT --log-dir LOG --map map.yaml
navig explore rename undo   ROOT --log-dir LOG --yes
```

Folders get English words from the map (`names:` any segment, `folders:` an exact path),
everything else a transliterated Latin slug; text files under `tag.globs` are prefixed with
their language and — only when the file states a date — its year
(`lyrics/Тексты/Выбежал из леса.md` → `lyrics/texts/ru--vybezhal-iz-lesa.md`). `words:` swaps
whole words inside new names (`kru: squad`); `keep:` protects already-Latin machine names.

`apply` moves file by file with an MD5 before and after, then **rewrites every reference** to
a moved path in the tree's text files — root-relative paths in either separator, absolute and
JSON-escaped absolute paths, markdown links and front-matter paths (resolved from where the
referencing file *was*, re-relativised from where it *is*) — backing up each text file before
its first rewrite. `rewrite_exclude:` keeps audit logs as they were. `verify` fails on any
missing file, MD5 change, leftover source, non-Latin path or old path still named in a text
file. Junctions and symlinks are never followed.

## Install

It ships in the NAVIG monorepo and loads automatically when NAVIG is installed.
To add/enable it explicitly:

```
navig store install navig-explore
```

Check wiring and toggle state:

```
navig store list
```

## Development

This package lives in the NAVIG monorepo under `plugins/navig-explore/`. It registers via
the `navig.plugins` entry-point group; CLI verbs register via `navig.commands`.
