Metadata-Version: 2.4
Name: vidxp
Version: 0.2.0
Summary: VidXP - Video indexing and search by dialogue, scene, and actor
Author: Muhammad Haroon, Talha Momin, Talha Amjad, Abdurrehman Subhani, Saad Bazaz
License-Expression: MIT
Project-URL: Repository, https://github.com/grayhatdevelopers/vidxp
Project-URL: Issues, https://github.com/grayhatdevelopers/vidxp/issues
Keywords: video,search,indexing,cli,streamlit
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.14,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: filelock>=3.13
Requires-Dist: packaging>=24
Requires-Dist: pydantic<3,>=2.8
Requires-Dist: rich
Requires-Dist: typer<1,>=0.27
Provides-Extra: storage
Requires-Dist: chromadb; extra == "storage"
Provides-Extra: dialogue
Requires-Dist: chromadb; extra == "dialogue"
Requires-Dist: moviepy==1.0.3; extra == "dialogue"
Requires-Dist: sentence-transformers<4,>=3.4; extra == "dialogue"
Requires-Dist: whisperx<3.9,>=3.8.6; extra == "dialogue"
Provides-Extra: scene
Requires-Dist: chromadb; extra == "scene"
Requires-Dist: clip-anytorch==2.6.0; extra == "scene"
Requires-Dist: numpy<3,>=2.1; extra == "scene"
Requires-Dist: opencv-python; extra == "scene"
Requires-Dist: Pillow>=7.0.0; extra == "scene"
Requires-Dist: setuptools<81; extra == "scene"
Requires-Dist: torch; extra == "scene"
Provides-Extra: actor
Requires-Dist: chromadb; extra == "actor"
Requires-Dist: face-recognition; extra == "actor"
Requires-Dist: numpy<3,>=2.1; extra == "actor"
Requires-Dist: opencv-python; extra == "actor"
Provides-Extra: all
Requires-Dist: chromadb; extra == "all"
Requires-Dist: moviepy==1.0.3; extra == "all"
Requires-Dist: sentence-transformers<4,>=3.4; extra == "all"
Requires-Dist: whisperx<3.9,>=3.8.6; extra == "all"
Requires-Dist: clip-anytorch==2.6.0; extra == "all"
Requires-Dist: numpy<3,>=2.1; extra == "all"
Requires-Dist: opencv-python; extra == "all"
Requires-Dist: Pillow>=7.0.0; extra == "all"
Requires-Dist: setuptools<81; extra == "all"
Requires-Dist: torch; extra == "all"
Requires-Dist: face-recognition; extra == "all"
Requires-Dist: numpy<3,>=2.1; extra == "all"
Requires-Dist: opencv-python; extra == "all"
Provides-Extra: frontend
Requires-Dist: streamlit>=1.37; extra == "frontend"
Provides-Extra: benchmarks
Requires-Dist: srt<4,>=3.5; extra == "benchmarks"
Dynamic: license-file

<p align="center">
<a href="https://github.com/grayhatdevelopers/vidxp">
<img alt="logo" src="https://github.com/grayhatdevelopers/vidxp/blob/main/docs/images/logo.png?raw=true" width="200"/>
</a>
</p>
<h1 align="center">VidXP</h1>
<p align="center">
<em>Search video by what was said, what appeared on screen, and recurring faces.</em>
</p>
<div align="center">
<p>
    VidXP is a local-first video indexing and search engine distributed as a Python
    package. 
  </p>
</div>
<hr/>
<br/>You can use it:
  <ul style="display:inline-block; text-align:left;">
<li>From the command line</li>
<li>Through its browser interface</li>
<li>As a desktop app</li>
<li>As an API</li>
<li>As an MCP, with your agents (coming soon ⚡️)
    <li>As an indexing and retrieval layer inside another application</li>
</li></ul>
<p align="center">
<strong>Dialogue search · Scene search · Actor grouping · CLI · Browser UI · Python API</strong>
</p>
<p align="center">
<a href="https://pypi.org/project/vidxp/">
<img alt="PyPI version" src="https://img.shields.io/pypi/v/vidxp"/>
</a>
<a href="https://pypi.org/project/vidxp/">
<img alt="Supported Python versions" src="https://img.shields.io/pypi/pyversions/vidxp"/>
</a>
<a href="https://github.com/grayhatdevelopers/vidxp/actions/workflows/ci.yml?query=branch%3Amain">
<img alt="CI status" src="https://github.com/grayhatdevelopers/vidxp/actions/workflows/ci.yml/badge.svg?branch=main&amp;raw=true"/>
</a>
<a href="https://github.com/grayhatdevelopers/vidxp/blob/main/LICENSE">
<img alt="MIT license" src="https://img.shields.io/github/license/grayhatdevelopers/vidxp"/>
</a>
<a href="https://grayhat.studio/discord">
<img alt="Discord" src="https://img.shields.io/discord/867124708473700363?logo=discord&amp;logoColor=white"/>
</a>
</p>

## Why VidXP

Finding one moment in a video should not require scrubbing through the entire
timeline. VidXP builds a searchable index from three kinds of evidence:

- **Dialogue:** semantic search over timestamped transcripts.
- **Scenes:** text-to-frame search.
- **Actors:** groups similar detected faces and exports a highlighted video for a selected cluster.

After the required model weights are available, video processing and search run completely locally, for your privacy and security.

Some ideas on how to use VidXP:
- Use it as way to find your favorite relatives in a huge folder of wedding videos (been there, done that)
- Use it in your application, allow users to search videos (an idea: use it alongside a video-editing application)
- Use it as an "understanding" layer so your LLM / agent can understand videos

[![Video Screenshot](https://github.com/grayhatdevelopers/vidxp/blob/main/docs/images/video-screenshot.jpeg?raw=true)](https://www.linkedin.com/feed/update/urn:li:activity:7343569473720725505/)


## Current capabilities

| Capability | Available now | Result |
|---|---|---|
| Dialogue search | Transcription, word alignment, semantic phrase indexing | Matching video time |
| Scene search | Text search over sampled video frames | Matching frame and time |
| Actor grouping | Within-video face detection and clustering | Clustered detections and highlighted output video |
| Interfaces | Typer CLI, Streamlit browser interface, Python API | Interactive or programmatic use |
| Index management | Saved progress, ready/failed state, cancellation, isolated programmatic runs | Traceable and reusable indexes |

## Quick start

VidXP supports Python 3.10 through 3.13 and requires FFmpeg. See the
[installation guide](https://github.com/grayhatdevelopers/vidxp/blob/main/INSTALLATION_GUIDE.md) for the `dlib` compiler
requirements, source installation, model preparation, and troubleshooting.

Install the command line and browser interface with
[pipx](https://packaging.python.org/en/latest/guides/installing-stand-alone-command-line-tools/).
The command is available on your `PATH` while VidXP and its dependencies remain
isolated:

```bash
pipx install "vidxp[all,frontend]"
```

Install only the capabilities you need with a smaller selection such as
`pipx install "vidxp[scene,frontend]"` or
`pipx install "vidxp[dialogue,scene]"`. To import VidXP from another Python
project, install it into that project's environment instead; the
[installation guide](https://github.com/grayhatdevelopers/vidxp/blob/main/INSTALLATION_GUIDE.md) covers that path.

Confirm the installed package and its runtime dependencies:

```bash
vidxp --version
vidxp doctor
```

The first use of each capability downloads its model weights. Download the fixed
dialogue, transcription, and scene models in advance with:

```bash
vidxp prepare
```

## Index and search

Build an index containing dialogue, scene, and actor information:

```bash
vidxp index create samplevideo.mp4
```

Search the completed index:

```bash
vidxp search dialogue "the bread just came out of the oven"
vidxp search scene "a yellow taxi on a city street" --top-k 5
vidxp actors list
vidxp actors render 1 samplevideo.mp4
```

Index only selected capabilities or sample fewer visual frames:

```bash
vidxp index create samplevideo.mp4 --modality scene --frame-stride 5
```

Repeat `--modality` to combine `dialogue`, `scene`, and `actor`.
Run `vidxp --help` or any command followed by `--help` for the complete command
reference.

Use named repositories to keep index locations and devices centrally
configured:

```bash
vidxp repositories add team --index-dir ./indexes/team --device cuda --use
vidxp repositories list
```

## Browser interface

Install the `frontend` extra and start:

```bash
vidxp ui
```

The command uses the active named repository, starts a local Streamlit server,
and remains active until stopped.
The interface can upload a video, start or cancel indexing, restore saved
progress after a page reload, and search the capabilities available in the
completed index.

## Container

Stable releases are available from GitHub Container Registry. Start the local
interface with persistent index and model storage by running:

```bash
docker compose up
```

See the [installation guide](https://github.com/grayhatdevelopers/vidxp/blob/main/INSTALLATION_GUIDE.md#run-the-container) for model
preparation, configuration, and direct `docker run` usage.

## Use VidXP as a Python package

The programmatic API supports isolated multi-video runs, supplied timestamped
transcripts, resumable per-video checkpoints, and metadata-rich top-k results.

```python
from vidxp.core import IndexConfig, VideoSource
from vidxp.core.runner import run_index
from vidxp.capabilities.scene.operations import search_scene

config = IndexConfig(
    dataset="my-library",
    split="local",
    run_id="demo",
    enabled_modalities=("scene",),
    frame_stride=5,
)

run_index(
    [
        VideoSource(video_id="video-1", path="videos/first.mp4"),
        VideoSource(video_id="video-2", path="videos/second.mp4"),
    ],
    config,
)

results = search_scene("a person enters a taxi", config=config, top_k=5)
for hit in results.hits:
    print(hit.video_id, hit.start, hit.end, hit.score)
```

The [Python indexing and retrieval contract](https://github.com/grayhatdevelopers/vidxp/blob/main/docs/benchmarking/core_contract.md)
documents configuration, stored metadata, result fields, and run layout.

## Recommended specs

&gt; Coming soon

---

## Roadmap

VidXP is an evolving beta. We'd love to hear your feedback and where you'd like to see the project go.

| Area | Current foundation | Direction |
|---|---|---|
| Search results | Top result in the CLI; structured top-k Python results | Rich ranked results, metadata, previews, and filtering across interfaces |
| Temporal search | Frame and transcript-phrase timestamps | Better time ranges, scene boundaries, aggregation, and ranking |
| Video collections | One local CLI/UI index; isolated multi-video Python runs | User-facing persistent multi-video libraries and index management |
| Actor workflows | Face clustering and highlighted video export | Cluster browsing, labeling, actor search, and stronger tracking |
| Speaker context | Timestamped dialogue search | Active-speaker detection and links between speech and visible people |
| Product experience | CLI and browser indexing/search | Clearer progress, result navigation, recovery, and long-running job controls |
| Evaluation | DiDeMo and HiREST baselines | Combined and component benchmarks, beginning with a LongVALE pilot |

## Models and local data

| Capability | Model |
|---|---|
| Dialogue embeddings | `sentence-transformers/all-MiniLM-L6-v2` |
| Transcription | WhisperX `large-v2` |
| Scene search | CLIP `ViT-B/32` |
| Word alignment | WhisperX model selected for the detected language |

VidXP maintains the standard local CLI/UI index in `chroma_data/`. Starting a
new local indexing run replaces the previous or incomplete local index. Model
caches normally live outside this directory and outside the virtual environment.

## Documentation and project links

- [Installation and troubleshooting](https://github.com/grayhatdevelopers/vidxp/blob/main/INSTALLATION_GUIDE.md)
- [Benchmarking status and results](https://github.com/grayhatdevelopers/vidxp/blob/main/docs/benchmarking/README.md)
- [Adding a capability](https://github.com/grayhatdevelopers/vidxp/blob/main/docs/adding-a-capability.md)
- [Changelog](https://github.com/grayhatdevelopers/vidxp/blob/main/CHANGELOG.md)
- [Issue tracker](https://github.com/grayhatdevelopers/vidxp/issues)
- [MIT license](https://github.com/grayhatdevelopers/vidxp/blob/main/LICENSE)


## Contributing

See [CONTRIBUTING.md](https://github.com/grayhatdevelopers/vidxp/blob/main/docs/CONTRIBUTING.md) for guidelines, maintainers, and how to submit PRs. AI/vibe-coded PRs welcome!

## Credits

Built by Grayhat Developers PVT Ltd. 2026. Maintained by the community.

Email: info@grayhat.studio

<a href="https://github.com/grayhatdevelopers/vidxp/graphs/contributors">
<img src="https://contrib.rocks/image?repo=grayhatdevelopers/vidxp"/>
</a>
