Metadata-Version: 2.4
Name: gemini-video-pipeline
Version: 0.1.0
Summary: Smart Video Chunker, Adaptive Ingestion Engine & Autonomous Agent Tool for Gemini
Author: Google Cloud Consulting - Global Solution Delivery
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/delta-fde/pipeline-video-chunker
Project-URL: Repository, https://github.com/delta-fde/pipeline-video-chunker
Project-URL: Issues, https://github.com/delta-fde/pipeline-video-chunker/issues
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: google-genai>=0.1.1
Requires-Dist: av>=11.0.0
Requires-Dist: pydantic>=2.5.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: click>=8.1.0
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: mypy>=1.6.0; extra == "dev"
Requires-Dist: flake8>=6.1.0; extra == "dev"
Provides-Extra: langchain
Requires-Dist: langchain-core>=0.1.0; extra == "langchain"
Dynamic: license-file

# gemini-video-pipeline

**Smart Video Chunker, Adaptive Ingestion Engine & Autonomous Agent Tool for Gemini**

`gemini-video-pipeline` is a production-grade Python library, CLI, and autonomous agent tool engineered to preprocess and chunk long-form videos deterministically, handing prepared, token-budgeted video chunks back to your client applications and Gemini agent workflows.

> [!NOTE]
> **Architecture Principle**: This library runs **100% locally and deterministically** for normalization, scene cutting, motion filtering, and token budgeting (0 LLM tokens spent). It prepares and yields chunk manifests and media files back to your client application, which then decides how to invoke Gemini models.
>
> 📖 **Looking for full documentation?** Check out the comprehensive **[User Guide](file:///Users/hemanthkrishna/Documents/video-chunker/USER_GUIDE.md)** for detailed API guides, agent integrations, surveillance use cases, and configuration options.

---

## Features

- **Container & Codec Normalization**: Seamlessly normalizes variable frame rate (VFR) and formats (MP4, MOV, MKV, WEBM, AVI) into standard constant-timebase ISO MP4 streams using PyAV without full re-encoding when possible.
- **Narrative Scene Detection**: Detects camera cuts via 32-bin HSV color histogram Chi-Square distance (`0.35` hard cuts, `0.20` soft cuts).
- **Motion Saliency Filtering**: Analyzes 64x64 luminance thumbnail frame differencing to eliminate idle/static camera intervals.
- **Token Budgeting Engine**: Enforces Gemini's multimodal video token budget (258 tokens/sec baseline), dynamically throttling FPS (0.2 to 2.0 FPS) and spatial resolution (1080p -> 720p/480p) per chunk.
- **Dual Transport & Ephemeral Staging**: For client applications targeting Gemini, automatically formats chunks `< 20 MB` inline via `types.Part.from_bytes()` and stages chunks `>= 20 MB` through the Gemini Files API (`client.files.upload()`) with active polling and context-managed cleanup.
- **Autonomous Agent Tooling**: Native Google ADK 2.x and LangChain structured tools for zero-LLM-cost deterministic video preprocessing that returns chunk manifests directly to your agent.

---

## Installation

### Option 1: Install from PyPI (Recommended)
```bash
pip install gemini-video-pipeline
```

### Option 2: Install Directly from GitHub
```bash
pip install "git+https://github.com/delta-fde/pipeline-video-chunker.git"
```

### Option 3: Local Editable Development
```bash
git clone https://github.com/delta-fde/pipeline-video-chunker.git
cd pipeline-video-chunker
pip install -e ".[dev]"
```
---

## Specifying the Chunks Output Directory & Disk Persistence

By default, the library writes chunks to `./chunks` and uses context managers to automatically clean up temporary `.mp4` chunks upon completion (`auto_cleanup=True`).

If you want chunk files saved to a custom path and preserved on disk:

### 1. In Python (`PipelineConfig`)
Pass `output_dir` (any relative or absolute path) and set `auto_cleanup=False`:

```python
from gemini_video_pipeline import VideoStreamPipeline, PipelineConfig

config = PipelineConfig(
    output_dir="/path/to/my_chunks",  # Custom chunks directory (created automatically if missing)
    auto_cleanup=False,               # Set to False to keep chunk files on disk
    target_token_budget_per_chunk=200_000,
    motion_filter_enabled=True,
    motion_threshold=0.10,
)

pipeline = VideoStreamPipeline(config=config)
for chunk in pipeline.process("input.mp4"):
    print(f"Chunk {chunk.chunk_index} saved at: {chunk.file_path}")
```

### 2. In Autonomous Agent Tooling (`VideoChunkerTool`)
Specify `output_dir` and `auto_cleanup` during tool initialization:

```python
from gemini_video_pipeline.tools.adk_tool import VideoChunkerTool

tool = VideoChunkerTool(
    output_dir="/path/to/my_chunks",  # Custom destination
    auto_cleanup=False                # Persist chunks on disk
)

# Run tool directly
manifest = tool.run({"video_path": "input.mp4"})
for item in manifest["active_chunks"]:
    print(f"Chunk {item['chunk']}: {item['file_path']}")
```

### 3. In the CLI (`--output-dir`)
Use the `--output-dir` option (defaults to `./chunks`):

```bash
gemini-video-pipeline chunk input.mp4 --output-dir /path/to/my_chunks --no-motion-filter
```

---

## CLI Usage

The CLI command is `gemini-video-pipeline`:

### 1. Probe Video Metadata
Inspect container format, codec, dimensions, FPS, VFR status, and duration:
```bash
gemini-video-pipeline probe video.mp4
```

### 2. Chunk Video onto Disk
Split a video into scene-aligned, token-budgeted MP4 files:
```bash
gemini-video-pipeline chunk video.mp4 \
  --output-dir /path/to/my_chunks \
  --budget 200000 \
  --motion-filter \
  --motion-threshold 0.10 \
  --max-duration 300.0
```

CLI options for `chunk`:
| Option | Default | Description |
|---|---|---|
| `--output-dir` | `./chunks` | Target folder for generated chunk files |
| `--budget` | `200000` | Maximum token ceiling per video chunk |
| `--motion-filter / --no-motion-filter` | `True` | Filter out static/idle intervals |
| `--motion-threshold` | `0.10` | Motion saliency cutoff threshold (0.0 to 1.0) |
| `--max-duration` | `300.0` | Maximum duration in seconds for any single chunk |

### 3. Analyze Video Chunks with Gemini
Splits video, uploads chunks, and queries Gemini with temporal grounding metadata (requires `GEMINI_API_KEY`):
```bash
export GEMINI_API_KEY="your-api-key"
gemini-video-pipeline analyze video.mp4 --prompt "Identify any security incidents" --model gemini-2.5-flash
```

---

## Python API Reference

### Pipeline Configuration Options (`PipelineConfig`)

```python
from gemini_video_pipeline import PipelineConfig

config = PipelineConfig(
    output_dir="./chunks",             # Directory where chunk files are saved
    auto_cleanup=True,                 # Ephemeral deletion of chunk files upon context exit
    target_token_budget_per_chunk=200_000, # Gemini token limit per chunk
    scene_cut_threshold=0.35,          # Chi-Square distance threshold for HARD scene cuts
    soft_cut_threshold=0.20,           # Chi-Square distance threshold for SOFT scene cuts
    motion_filter_enabled=True,        # Whether to skip static/idle footage
    motion_threshold=0.10,             # Mean luminance delta threshold for idle detection
    max_chunk_duration_sec=300.0,      # Max duration before forcing a chunk boundary
    min_chunk_duration_sec=2.0,        # Min chunk duration to prevent micro-fragments
    sample_rate_fps=1.0,               # Base frame sampling rate
    upload_to_files_api=True,          # Route chunks >= 20 MB through Gemini Files API
)
```

### Chunk Object Attributes (`VideoChunk`)

Each chunk yielded by `pipeline.process(video_path)` contains:
- `chunk.chunk_index`: 1-based index of the chunk.
- `chunk.file_path`: Local disk path (inside `output_dir`).
- `chunk.start_timestamp` / `chunk.end_timestamp`: Global timestamps formatted as `HH:MM:SS.mmm`.
- `chunk.start_sec` / `chunk.end_sec`: Start and end times in seconds.
- `chunk.duration_sec`: Duration of this chunk.
- `chunk.frame_count`: Total video frames in the chunk.
- `chunk.source_fps`: Source frame rate.
- `chunk.average_motion_score`: Average motion saliency score (`0.0` to `1.0`).
- `chunk.estimated_tokens`: Estimated Gemini multimodal token consumption.
- `chunk.sampling_plan`: Token budget plan with `target_fps` and optional `scale_resolution`.
- `chunk.to_gemini_content()`: Converts the chunk directly into a `types.Part` (inline bytes or Files API URI).

---

## Autonomous Agent Integration

### Google ADK 2.x
```python
from gemini_video_pipeline.tools.adk_tool import VideoChunkerTool

tool = VideoChunkerTool(output_dir="./agent_chunks", auto_cleanup=False)

# Register as function declaration for Gemini function calling:
func_declaration = tool.as_function_declaration()

# Or pass directly as an ADK callable:
agent_tool = tool.as_adk_callable()
manifest = agent_tool(
    video_path="security_cam.mp4",
    token_budget_per_chunk=150_000,
    focus_high_motion_only=True
)
```

### LangChain
```python
from gemini_video_pipeline.tools.langchain_tool import create_langchain_video_chunker_tool

tool = create_langchain_video_chunker_tool(output_dir="./langchain_chunks")
```

---

## Testing & Verification

1. **Run Unit and Integration Tests**:
   ```bash
   pytest tests/ -v --cov=gemini_video_pipeline --cov-report=term-missing
   ```

2. **Run Linting and Static Type Checking**:
   ```bash
   flake8 src tests --max-line-length=120
   mypy src
   ```

3. **Run Self-Contained Examples**:
   ```bash
   python examples/01_direct_library_usage.py
   python examples/02_motion_filtered_audit.py
   python examples/03_adk_agent_workflow.py
   ```

4. **Test Across All Formats, Codecs & Lengths (including >10min and ~30min)**:
   ```bash
   # Generate standard multi-format test videos (MP4, MOV, MKV, WEBM, AVI, 3s to 3min, VFR, CCTV)
   python scripts/generate_test_videos.py

   # Generate long test videos (12-minute multi-scene video and 30-minute narrative video)
   python scripts/generate_long_videos.py

   # Process and verify all 12 test videos with output chunks on disk
   python scripts/test_all_videos.py
   ```

5. **Test Variable Token Budgets & Chunk Duration Constraints**:
   Run the verification matrix across a spectrum of token budgets (`5k` to `500k` tokens) and chunk duration limits (`15s` to `240s`):
   ```bash
   python scripts/test_variable_budgets_and_chunks.py
   ```
