Metadata-Version: 2.4
Name: clipsmith-ai
Version: 0.2.1
Summary: Twitch VOD -> AI-selected vertical clips with burned-in Spanish captions.
Project-URL: Homepage, https://github.com/ricardogr07/clipsmith
Project-URL: Repository, https://github.com/ricardogr07/clipsmith
Project-URL: Documentation, https://ricardogr07.github.io/clipsmith
Author-email: Ricardo García Ramírez <rgr5882@gmail.com>
License: MIT
License-File: LICENSE
Keywords: ai,clips,ffmpeg,llm,twitch,vertical-video,whisper
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: End Users/Desktop
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: anthropic>=0.40
Requires-Dist: chat-downloader>=0.2
Requires-Dist: faster-whisper>=1.0
Requires-Dist: httpx>=0.27
Requires-Dist: openai>=1.40
Requires-Dist: pydantic-settings>=2.2
Requires-Dist: pydantic>=2.6
Requires-Dist: python-dotenv>=1.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.7
Requires-Dist: twitch-dl>=3.3
Requires-Dist: typer>=0.12
Provides-Extra: cloud
Requires-Dist: azure-identity>=1.16; extra == 'cloud'
Requires-Dist: azure-mgmt-containerinstance>=10; extra == 'cloud'
Requires-Dist: azure-mgmt-resource>=23; extra == 'cloud'
Requires-Dist: azure-mgmt-storage>=21; extra == 'cloud'
Requires-Dist: azure-storage-file-share>=12; extra == 'cloud'
Requires-Dist: google-api-python-client>=2; extra == 'cloud'
Requires-Dist: google-auth-oauthlib>=1; extra == 'cloud'
Provides-Extra: dev
Requires-Dist: bandit>=1.7; extra == 'dev'
Requires-Dist: mkdocs-material>=9.5; extra == 'dev'
Requires-Dist: mkdocs>=1.6; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pip-audit>=2.7; extra == 'dev'
Requires-Dist: pre-commit>=3.7; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=9.0.3; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Provides-Extra: ollama
Requires-Dist: ollama>=0.3; extra == 'ollama'
Provides-Extra: vision
Requires-Dist: opencv-python-headless>=4.9; extra == 'vision'
Description-Content-Type: text/markdown

# clipsmith

Local Twitch → AI clip pipeline. Downloads a VOD, transcribes Spanish audio, ranks moments by chat activity, sends candidates to an LLM, and cuts 9:16 vertical MP4s — optionally with burned-in captions and a stacked webcam/gameplay layout for TikTok / YouTube Shorts.

## Prerequisites

| Tool | Install |
|------|---------|
| Python 3.11+ | python.org or `winget install Python.Python.3` |
| ffmpeg | `winget install Gyan.FFmpeg` — must be on `PATH` |
| twitch-dl | bundled via `pip install -e .` |
| chat-downloader | bundled via `pip install -e .` |
| faster-whisper | bundled via `pip install -e .` |

Optional — webcam auto-detection:
```sh
pip install -e ".[vision]"   # installs opencv-python-headless
```

## Installation

```sh
git clone https://github.com/ricardogr07/clipsmith
cd clipsmith
pip install -e .
```

## Configuration

### Secrets — `.env`

Copy `.env.example` to `.env` and fill in your keys:

```
TWITCH_CLIENT_ID=...
TWITCH_CLIENT_SECRET=...
ANTHROPIC_API_KEY=...       # if using provider: anthropic
OPENAI_API_KEY=...          # if using provider: openai
```

Get Twitch credentials at <https://dev.twitch.tv/console> → Register Your Application → OAuth Redirect: `http://localhost`.

### Behaviour — `config.yaml`

```yaml
channels:
  - chuyelwuero             # Twitch logins to watch

llm:
  provider: anthropic       # anthropic | openai | ollama
  model_anthropic: claude-sonnet-4-6

clip:
  min_seconds: 15
  max_seconds: 30

caption:
  enabled: false            # set true to burn subtitles into the video
  font: Arial
  font_size: 72             # ASS pts at 1080×1920 PlayRes
  outline: 3
  position: bottom          # bottom | middle | top

reframe:
  # center | webcam | stacked | none
  mode: none
  # Source-pixel crop for the webcam/face panel [x, y, w, h]
  # Leave null to use center-crop fallback; set once per stream layout.
  webcam_rect: null
  # Source-pixel crop for the gameplay panel (stacked mode only)
  # null = center-crop fallback
  gameplay_rect: null
  # Fraction of 1920px height given to the top (webcam) panel in stacked mode
  # e.g. 0.4 → 768px webcam on top, 1152px gameplay on bottom
  split_ratio: 0.4
```

## Usage

### First-time setup

```sh
clipsmith setup
```

Saves your API key to `.env` and verifies ffmpeg is available.

### Process a local MP4

```sh
clipsmith process path/to/recording.mp4
```

Runs the full pipeline — transcribe → score candidates → LLM selection → cut clips. Clips appear in `out/<video_id>/`.

Useful flags:

| Flag | Effect |
|------|--------|
| `--skip-transcribe` | Load cached `transcript.json` |
| `--captions / --no-captions` | Override caption config |
| `--reframe / --no-reframe` | Override reframe config |
| `--provider anthropic\|openai` | Override LLM provider |

### Daemon mode

```sh
clipsmith watch
```

Polls every `poll_interval_s` seconds (default 120). When a new archive VOD appears it runs the full pipeline automatically. State is persisted to `state.json` so already-processed VODs are skipped across restarts.

### One-off Twitch VOD

```sh
clipsmith run-vod <video_id>
```

Useful flags:

| Flag | Effect |
|------|--------|
| `--skip-download` | Use the existing `.mp4` in `work/<id>/` |
| `--skip-transcribe` | Load cached `transcript.json` |
| `--skip-chat` | Load cached `chat.json` |
| `--skip-select` | Stop after candidates, skip LLM |
| `--skip-clip` | Stop after picks, skip ffmpeg |
| `--provider anthropic\|openai` | Override config LLM |
| `--max-candidates N` | Cap candidates sent to LLM (default 20) |

### Re-cut flat clips

Re-run only the ffmpeg step from an existing `picks.json` (e.g. after adjusting caption style):

```sh
clipsmith clip <video_id>
```

### Stacked vertical layout (webcam + gameplay)

After reviewing the flat clips, pick the best ones and reframe them into a stacked 9:16 layout — webcam on top, gameplay on bottom:

```sh
clipsmith reframe <video_id> clip_01 clip_04 clip_09
```

Output goes to `out/<video_id>/stacked/`. The reframe rects come from `config.yaml`; see the `reframe` section above.

**Auto-detecting the webcam rect:**

If `reframe.webcam_rect` is not set in `config.yaml`, clipsmith will try to detect the face rectangle automatically using OpenCV (requires `pip install -e ".[vision]"`). The result is cached to `work/<video_id>/webcam_rect.json` and reused on subsequent runs.

To run detection manually and write the result directly into `config.yaml`:

```sh
clipsmith detect-webcam <video_id>
```

This samples frames, detects the webcam rectangle, and saves `reframe.webcam_rect` in `config.yaml` automatically — no manual copy-paste needed.

### Sanity check

```sh
clipsmith whoami chuyelwuero
```

## Output

```
work/
  <video_id>/
    <video_id>.mp4          downloaded (or copied) source VOD
    transcript.json         faster-whisper segments + word timestamps
    chat.json               chat replay
    candidates.json         ranked candidate moments
    picks.json              LLM-accepted clips with titles and reasons
    webcam_rect.json        auto-detected webcam rect (cached, vision only)

out/
  <video_id>/
    clip_01_momento_gracioso.mp4    flat 9:16 clip
    clip_01_momento_gracioso.ass    sidecar ASS subtitle file
    clip_02_reaccion_epica.mp4
    ...
    stacked/
      clip_01_momento_gracioso.mp4  stacked layout (webcam top, gameplay bottom)
      clip_04_reaccion_epica.mp4
      ...
```

All clips are 1080×1920 (9:16), 15–30 s, h264/aac, with optional burned-in Spanish karaoke captions.

## Pipeline overview

```
watch / run-vod / process
  ↓
downloader      (twitch-dl)                → work/<id>/<id>.mp4
  ↓
detect-webcam   (opencv Haar cascade)      → work/<id>/webcam_rect.json  [optional]
  ↓
transcribe      (faster-whisper)           → transcript.json  (word timestamps, lang=es)
  ↓
chat            (chat-downloader)          → chat.json
  ↓
candidates      (density + clips + !clip)  → candidates.json
  ↓
selector        (LLM)                      → picks.json
  ↓
clipper         (ffmpeg + libass)          → out/<id>/clip_NN_<title>.mp4
  ↓  [manual: clipsmith reframe]
reframe         (ffmpeg filter_complex)    → out/<id>/stacked/clip_NN_<title>.mp4
```

## Development

```sh
pip install -e ".[dev]"
pip install -e ".[vision]"   # optional, for detect-webcam tests
python -m pytest tests -q
```
