Metadata-Version: 2.4
Name: shortsmaker
Version: 0.1.0
Summary: Turn a long interview into publish-ready vertical clips: cut on sentence boundaries, reframed to 9:16, karaoke subtitles burned in.
Project-URL: Homepage, https://github.com/pirocheto/shortsmaker
Project-URL: Repository, https://github.com/pirocheto/shortsmaker
Project-URL: Issues, https://github.com/pirocheto/shortsmaker/issues
Author-email: pirocheto <pirocheto@gmail.com>
License: MIT
License-File: LICENSE
Keywords: ai,clips,reframe,shorts,subtitles,transcription,vertical-video,video
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Video
Requires-Python: >=3.12
Requires-Dist: google-genai>=1.0
Requires-Dist: mediapipe>=0.10.35
Requires-Dist: numpy>=2.0
Requires-Dist: openai>=1.60
Requires-Dist: opencv-python-headless>=4.10
Requires-Dist: pydantic-settings>=2.7
Requires-Dist: pydantic>=2.10
Requires-Dist: rich>=13.9
Requires-Dist: typer>=0.15
Description-Content-Type: text/markdown

<div align="center">

# 🎬 shortsmaker

**A long interview goes in. Publish-ready vertical clips come out.**

Cut on sentence boundaries · reframed to 9:16 by body tracking · karaoke subtitles burned in

[![Python 3.12+](https://img.shields.io/badge/python-3.12+-3776AB?style=for-the-badge&logo=python&logoColor=white)](https://www.python.org/)
[![OpenAI Whisper](https://img.shields.io/badge/openai-whisper-412991?style=for-the-badge&logo=openai&logoColor=white)](https://platform.openai.com/docs/guides/speech-to-text)
[![Google Gemini](https://img.shields.io/badge/google-gemini-8E75B2?style=for-the-badge&logo=googlegemini&logoColor=white)](https://ai.google.dev/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue?style=for-the-badge)](LICENSE)

</div>

```bash
shortsmaker run interview.mp4
```

A 15-minute extract of a podcast becomes four or five clips you could post, in about five minutes,
on a laptop with no GPU. French by default (`-l fr`); also handles English, Spanish, German,
Italian, Portuguese and Dutch.

### What a finished clip looks like

**▶ [Watch an example clip](docs/example-clip.mp4)** — 21 seconds, straight out of `shortsmaker run`,
not retouched.

It shows the three things the pipeline exists to get right. The clip **opens and closes on a whole
sentence**, because its bounds were looked up in a word-level transcript rather than guessed. The
**crop holds on the speaker** and jumps only where the show's own editor cut — it never drifts, which
would invent a camera move that isn't in the source. And the **caption highlights each word as it is
spoken**, timed from the same transcript, with the title written by the model at the top.

---

## Install

Needs **Python 3.12+**, [uv](https://docs.astral.sh/uv/), and **FFmpeg built with libass** (the
distro package is fine — `apt install ffmpeg`).

### Quick start (from PyPI)

```bash
# Run directly without installation
uvx shortsmaker run interview.mp4

# Or install globally
uv tool install shortsmaker
shortsmaker run interview.mp4
```

One install, both uses: the `shortsmaker` command, and an importable pipeline. **Importing it does
not turn it into a CLI** — nothing writes to stdout, and the stages log through the standard
`logging` module, silent until your application configures a handler:

```python
from shortsmaker import Paths, Settings
from shortsmaker.stages import analyze, transcribe

transcript = transcribe.transcribe(paths, Settings())
segments = analyze.analyze(paths, transcript, Settings(), clips=3)
```

See [the package docstring](src/shortsmaker/__init__.py) for the full loop — and note the
`if __name__ == "__main__":` guard it insists on, which the reframe's worker processes require.

The font and the pose model are vendored **inside** the package, so the install runs offline with
nothing else to fetch.

### Development install (from source)

```bash
git clone … && cd shortsmaker
uv sync
cp .env.example .env      # then fill in the two keys
```

### API Keys

| Key | For |
|---|---|
| `OPENAI_API_KEY` | Whisper transcription |
| `GEMINI_API_KEY` | Choosing the segments |

Both are metered, paid APIs — `transcribe` and `analyze` are the only two stages that call out to
them, everything else runs locally and free.

**Testing without API keys?** Use `--mock` to simulate LLM responses:
```bash
shortsmaker run interview.mp4 --mock  # No API calls, no cost
```

---

## Use

```bash
shortsmaker run interview.mp4                    # everything → clips/
shortsmaker run interview.mp4 -n 3               # keep only the 3 best passages
shortsmaker run interview.mp4 -f 1:1 -l en       # square, English
shortsmaker run interview.mp4 --mock             # test without API calls
```

| | | |
|---|---|---|
| `-f` `--format` | `9:16` | `9:16` TikTok/Reels/Shorts · `4:5` Instagram feed · `1:1` LinkedIn/X · `16:9` keep the original shape (nothing is cropped, and the tracking is skipped entirely) |
| `-l` `--language` | `fr` | The language spoken — and the language the titles and summaries are written in. |
| `-n` `--clips` | all | Keep only the N best-scoring passages. |
| `--min-score` | none | Keep only the passages scoring at least this. Combines with `-n`. |
| `-j` `--jobs` | `0` (auto) | Clips processed at once. `0` picks a sensible number from the machine's core count. |
| `-w` `--work-dir` | `.shortsmaker/` beside the video | Where intermediates (transcript, segments, crop plans) are kept. |
| `-o` `--out` | `clips/` beside the video | Where the finished clips land. |
| `-v` `--verbose` | off | Show the underlying ffmpeg commands and the raw API traffic. |
| `--mock` | off | Simulate LLM responses for testing without API keys or costs. |

`--force` has **no short flag on purpose**: it re-runs the paid stages, so it should cost a moment's
thought.

Or one stage at a time. **Only the first two cost money**, and they cache — so you can re-run the
last two as often as you like while tuning the look:

```bash
shortsmaker transcribe interview.mp4        # Whisper   → .shortsmaker/transcript.json   (the clock)
shortsmaker analyze    interview.mp4        # Gemini    → .shortsmaker/analysis.json     (paid, kept whole)
shortsmaker show       interview.mp4        # just print what was found, with scores
shortsmaker reframe    interview.mp4        # MediaPipe → .shortsmaker/reframe/
shortsmaker render     interview.mp4        # ffmpeg    → clips/

shortsmaker render interview.mp4 --size 96 --accent "#00E5FF"   # restyle, free
shortsmaker render interview.mp4 --only 3                       # re-render just clip 3, leave the rest
shortsmaker analyze interview.mp4 --prompt-file mine.txt        # your own prompt instead of the built-in one
```

There is no `cut` command. Cutting the clips out of the source is free, fast and required before
anything else can run, so it is not a decision to make: `reframe` and `render` do it themselves, and
skip it when the clips are already there.

**Choosing which passages to keep costs nothing.** `analyze` writes everything it found to
`analysis.json` and never narrows it; `-n` and `--min-score` select from that into `segments.json`,
which is what the rest of the pipeline reads. So changing your mind is free — no API call is made:

```bash
shortsmaker analyze interview.mp4 -n 3            # keep the 3 best…
shortsmaker analyze interview.mp4 --min-score 0.8 # …no, the ones that scored 0.8+ — still free
```

`render` does not ask for the format: the crop plan records what it was built for, so it reads it
back. Pass `-f` only to re-target.

`shortsmaker --help` on anything. The defaults carry their reasoning: `--pop` explains why 100 means *no*
zoom, `--shadow` warns that ASS's shadow is a hard offset copy rather than a soft one.

`shortsmaker run` and `shortsmaker analyze` print a token table after the Gemini call — prompt, output and total
tokens for the run.

---

## Offline testing with `--mock`

You can test the pipeline offline without making any API calls by using the `--mock` flag. This replaces LLM responses with realistic mock data:

```bash
shortsmaker run interview.mp4 --mock              # full pipeline with mock responses
shortsmaker transcribe interview.mp4 --mock       # mock Whisper transcription
shortsmaker analyze interview.mp4 --mock          # mock Gemini analysis
```

**When to use it:**
- Testing the pipeline structure without spending credits
- Developing locally without API access
- Verifying the downstream stages (cut, reframe, render) work correctly
- CI/CD pipelines and automated testing

**What gets mocked:**
- Whisper transcription: generates realistic word-level timestamps and punctuated sentences
- Gemini analysis: produces segment proposals with scores, labels, and quotes from the mock transcript

The mock data is deterministic but randomized enough to test edge cases. Once transcription and analysis are cached (or mocked), the remaining stages run exactly as they would with real API data.

---

## How it works, and why it looks like this

**→ [SPEC.md](SPEC.md)** — the flowchart, and the reasoning, stage by stage. Four spikes were built
and measured before any of this was written, and nearly every rule in the code replaced one that
made the clips worse.

The three decisions worth knowing:

**Gemini decides *what* to cut. Whisper decides *when*.** Gemini reliably picks the right moments and
cannot place them in time — its timestamps drift by ~2 s, and one run answered in `MM.SS` while the
prompt demanded seconds, producing five 1-second clips that passed every check in silence. So it is
never asked for timestamps. It **quotes the words**, and the quote is looked up in Whisper's
word-level transcript. Bounds land on real word boundaries by construction. Which is why
**transcription runs first**: it is the clock, not a subtitle afterthought.

**Gemini gets the audio, never the video.** The source is a talking head. The frames carry nothing
the audio doesn't, but at ~1 fps they dominate the token bill *and* are the only reason the context
window would ever overflow. The signal that decides whether a Short lands — a laugh, the pause before
a punchline, a voice dropping — is in the audio, and Gemini hears it natively. 56 MB of video becomes
4.5 MB of audio.

**The reframe tracks bodies, and never guesses who is speaking.** Face detection was tried and failed
exactly where it mattered: on a wide shot it missed the guest (in profile, head down) and
hallucinated a box on the background. Bodies don't vanish in profile. And when two people are in
frame, we don't pick a speaker — we show both, on a blurred fill. A question you decline to ask has
no wrong answer.

---

## Performance

Measured on a 16-core CPU, **no GPU**:

| | |
|---|---|
| Reframe (MediaPipe) | 0.71× realtime |
| Render (crop + blur + libass + H.264) | 0.45× realtime |
| **Local processing** | **~1.2× realtime** |

Transcription and analysis are network-bound: about 90 seconds for a 15-minute video.

---

## Development

```bash
uv sync                                            # runtime deps + pytest
uv run pytest                                      # unit tests

uvx ruff@0.15.21 check .                           # lint
uvx ty@0.0.58 check                                # type check
uvx bandit@1.9.4 -c pyproject.toml -r src          # security
```

These are, verbatim, the four commands CI runs — see [ci.yml](.github/workflows/ci.yml). Green here
means green there.

The three tools are **not** in `pyproject.toml`'s dev group, and that is deliberate: they read the
source, they are never imported by it, and locking them would pin their versions in a second place
that nothing keeps in step with CI. The pins live in `ci.yml`, once. `pytest` *is* a dev dependency,
because the test code imports it.

Rules deliberately off: `E501` (the formatter is not run — the ffmpeg argument blocks are aligned by
hand and `ruff format` would explode them one argument per line) and `RUF001-003` (the prose uses
`×`, not `x`, and that is not a typo). Bandit skips four checks, each justified in `pyproject.toml`
— chiefly the `subprocess` calls that *are* the product. `B602` (`shell=True`) stays on.

To publish a release, see **[PUBLISHING.md](PUBLISHING.md)**. The short version: push a tag.

---

## Licence and credits

The code is MIT — see [LICENSE](LICENSE).

**The example clip in `docs/` is not.** It is a short excerpt of an interview with Bernard Arnault
from the show *Legend* (Guillaume Pley), reproduced here for the sole purpose of illustrating what
the tool outputs. All rights to the footage remain with its owners; it is not covered by this
project's licence, and it is not redistributable as part of it.

The vendored assets carry their own licences: **Lato** (Łukasz Dziedzic, SIL Open Font License 1.1)
and MediaPipe's **pose_landmarker** (Google, Apache 2.0).
