Metadata-Version: 2.5
Name: sub-tools
Version: 0.9.2
Summary: A robust Python toolkit powered by Google's Gemini API for converting video content into accurate, multilingual subtitles.
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: audioop-lts>=0.2.1; python_version >= '3.13'
Requires-Dist: google-api-core>=2.28.1
Requires-Dist: google-genai>=1.52.0
Requires-Dist: openai>=1.68.0
Requires-Dist: pycountry>=24.6.1
Requires-Dist: pytest-asyncio>=0.23.5
Requires-Dist: pytest>=9.0.1
Requires-Dist: rich>=13.9.4
Requires-Dist: subtitle-edit-rate==0.4.0
Description-Content-Type: text/markdown

# sub-tools 🎬

[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A toolkit for multilingual subtitles. A model transcribes the audio and translates the result; every answer is repaired and checked before it is accepted. Gemini 3.7 Flash is the primary model; OpenAI models such as GPT-5.6 Luna are supported as a lower-cost alternative.

## ✨ Features

- 🎯 Transcription straight from audio to SRT, with Gemini or an OpenAI model
- 🧰 Automatic repair of malformed model output, with a retry when repair cannot save it
- ✅ Strict validation that refuses to ship a broken subtitle file
- 🌍 Multilingual translation that preserves the source timings
- 🔊 Dubbing: subtitles spoken back into a timing-aligned MP3 per language
- 📥 Support for HLS streams, direct file URLs, and local files
- 🎵 Audio fingerprinting using Shazam (macOS only)
- 📊 Progress tracking with rich terminal output

## 🚀 Quick Start

### Prerequisites

- Python 3.10 or higher
- [FFmpeg](https://ffmpeg.org/) installed on your system

### Installation

```shell
pip install sub-tools
```

For a source checkout, run the setup script from a shell. It installs `uv` when it is
missing, provisions a supported Python interpreter if needed, and syncs the project
environment:

```shell
./setup.sh
```

### Usage

```shell
export GEMINI_API_KEY={your_api_key}

# Full pipeline: download video, extract audio, transcribe, and translate
sub-tools -i https://example.com/video.mp4 --languages en es fr

# Using HLS stream URL
sub-tools -i https://example.com/hls/video.m3u8 --languages en es fr

# Using local audio file (skip video/audio tasks)
sub-tools --tasks transcribe translate --audio-file audio.mp3 --languages en es fr

# Only transcribe without translation
sub-tools --tasks transcribe --audio-file audio.mp3 --languages en

# Dub existing subtitles only (uses the {language}.srt files already in the output directory)
sub-tools --tasks dub --audio-file audio.mp3 --languages es fr

# Specify custom tasks (available: video, audio, signature, transcribe, translate, dub)
sub-tools -i https://example.com/video.mp4 --tasks video audio transcribe translate --languages en es

# Specify a custom Gemini model for transcription and translation
sub-tools -i https://example.com/video.mp4 --languages en --model gemini-3.6-flash

# Use an OpenAI model instead of Gemini (reads OPENAI_API_KEY)
export OPENAI_API_KEY={your_api_key}
sub-tools -i https://example.com/video.mp4 --languages en es --model gpt-5.6-luna

# Dub: speak the translated subtitles into es.mp3 and fr.mp3
sub-tools --tasks transcribe translate dub --audio-file audio.mp3 --languages es fr

# Specify output directory (default: output)
sub-tools -i https://example.com/video.mp4 --languages en --output my-subtitles
```

### Choosing a provider

The provider follows from the model name: `gpt-*` models call the OpenAI API with
`OPENAI_API_KEY` (or `--openai-api-key`), everything else calls the Gemini API with
`GEMINI_API_KEY` (or `--gemini-api-key`). The same repair and validation loop runs
either way.

OpenAI text models such as `gpt-5.6-luna` cannot hear audio, so transcription is
routed to an audio-capable model (`whisper-1` on the transcription API by default;
override with `--audio-model`, which also accepts `gpt-audio-*` chat models) while
the selected model handles translation text-only. Audio
sent to OpenAI is inlined into the request; files over 15 MB are automatically
re-encoded to mono 32 kbit/s MP3, which fits roughly an hour of speech under the
20 MB request cap.

### Pipeline Tasks

The tool operates as a multi-stage pipeline controlled by the `--tasks` parameter:

1. **video**: Downloads media from URL (HLS or direct) → `video.mp4`
2. **audio**: Extracts audio track → `audio.mp3`
3. **signature**: Generates Shazam signature for fingerprinting (macOS only)
4. **transcribe**: The model turns the audio into subtitles → `{source-language}.srt`
5. **translate**: The model translates those subtitles into each target language → `{language}.srt`
6. **dub**: Text-to-speech speaks each `{language}.srt` into a timing-aligned `{language}.mp3`

By default, all tasks except `dub` run. You can customize which tasks to run with `--tasks`.

### Dubbing

Each subtitle cue is spoken by the provider's text-to-speech model (OpenAI:
`gpt-4o-mini-tts`, Gemini: `gemini-2.5-flash-preview-tts`; override with
`--tts-model` and `--tts-voice`) and placed at the cue's start time over silence,
producing an MP3 the same length as the original recording. Speech that runs longer
than the original speaker took is sped up (at most 2×) rather than talking over the
next cue, and `[sound effects]` are not spoken. The dub uses the same provider as
`--model`.

## 📏 Transcription evaluation

The evaluator is deliberately separate from model execution: it scores generated SRT
files against a human reference so multiple runs can be compared on identical input.
The primary score is the published [SubER method](https://aclanthology.org/2022.iwslt-1.1/),
implemented by the pinned [`subtitle-edit-rate==0.4.0`](https://pypi.org/project/subtitle-edit-rate/)
package. SubER is reference-based and accounts for subtitle text, segmentation, and
timing; lower is better. It is a published academic method and reference
implementation, not an NIST certification.

The report also includes the package's automatic-segmentation lexical metrics for
SRTs with different cue boundaries: `AS-WER`, `AS-CER`, `AS-BLEU`, `AS-TER`, and
`AS-chrF`. `AS-WER` and `AS-CER` use the same substitution/insertion/deletion
edit-rate convention documented in [NIST SCTK/SCLITE](https://github.com/usnistgov/SCTK/blob/master/doc/sclite.htm);
BLEU, TER, and chrF are provided by SacreBLEU through `subtitle-edit-rate`.
The report also includes the package's timing-aligned `t-WER`, `t-CER`, `t-BLEU`,
`t-TER`, and `t-chrF` diagnostics. This command does not invoke SCTK itself. The
evaluator is intentionally package-only: it does not add a custom score, timing
metric, coverage metric, or release gate.

The package documents the AS alignment family as the established automatic
segmentation approach (Matusov et al., [IWSLT 2005](https://aclanthology.org/2005.iwslt-1.19/))
and the t-BLEU timing-alignment approach (Cherry et al.,
[Interspeech 2021](https://www.isca-archive.org/interspeech_2021/cherry21_interspeech.pdf)).
The implementation here calls the package APIs directly; it does not reimplement
either alignment or any metric.

SubER is the primary score because it is the package's timing- and
segmentation-aware metric. AS-WER, AS-CER, and AS-TER are error rates (lower is
better); AS-BLEU and AS-chrF are similarity scores (higher is better). BLEU can be
low on very short samples because it requires n-gram matches, so it should be read
alongside the other metrics rather than used alone. The `t-*` metrics re-segment
the hypothesis using subtitle timings; they are supplemental diagnostics and do
not replace SubER's joint timing/segmentation score.

Give each hypothesis a stable name with `NAME=PATH`; repeat `--hypothesis` to compare
models or pipeline stages:

```shell
sub-tools-eval \
  --reference reference/en.srt \
  --hypothesis gemini-3.7-flash=output/gemini-3.7-flash/en.srt \
  --hypothesis gemini-3.6-flash=output/gemini-3.6-flash/en.srt \
  --output evals/transcription.json \
  --markdown evals/transcription.md
```

Only the reference and generated SRT files are inputs; no audio file or API key is
required. Private or copyrighted recordings are intentionally not bundled in the
package.

`sub-tools-eval` measures the text, segmentation, and timing quality of the assembled
SRT output, while `sub-tools` remains responsible for producing the SRT.

To compare models or settings, pass each generated file as a `--hypothesis`. The
Markdown report shows one row per variant; lower error rates and higher BLEU/chrF
indicate a closer match to the reference.

For the reproducible evaluation harness and commands, see
[evals/README.md](evals/README.md).

### Build Docker

```shell
docker build -t sub-tools .
docker run -v $(pwd)/output:/app/output sub-tools sub-tools --gemini-api-key GEMINI_API_KEY -i URL -l en
```

## 🤝 Contributing

Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.

### Quick Development Setup

```shell
# Clone and setup
git clone https://github.com/dohyeondk/sub-tools.git
cd sub-tools
./setup.sh  # installs uv and runs uv sync
```

## 🧪 Testing

```shell
uv run pytest -m "not slow"
```

The evaluation metrics have unit tests in `tests/test_evaluation.py` and do not require
an API key or an audio file.

## 📝 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.


## ⭐ Star History

[![Star History Chart](https://api.star-history.com/svg?repos=dohyeondk/sub-tools&type=Date)](https://star-history.com/#dohyeondk/sub-tools&Date)
