Metadata-Version: 2.5
Name: subxx
Version: 0.5.0
Summary: YouTube transcript / subtitle fetching toolkit for Python. Provides command line and HTTP interfaces. Supports language selection, SRT or VTT output
Project-URL: Homepage, https://gist.github.com/cprima/subxx
Project-URL: Documentation, https://gist.github.com/cprima/subxx
Project-URL: Repository, https://gist.github.com/cprima/subxx
Author-email: Christian Prior-Mamulyan <cprior@gmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: api,cli,subtitles,youtube,yt-dlp
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Video
Requires-Python: >=3.9
Requires-Dist: beautifulsoup4>=4.14.2
Requires-Dist: curl-cffi>=0.5.0
Requires-Dist: pydantic>=2.12
Requires-Dist: tomli>=2.0.0; python_version < '3.11'
Requires-Dist: typer>=0.9.0
Requires-Dist: yt-dlp>=2023.3.4
Provides-Extra: api
Requires-Dist: anyio>=3.7.0; extra == 'api'
Requires-Dist: fastapi>=0.100.0; extra == 'api'
Requires-Dist: uvicorn[standard]>=0.23.0; extra == 'api'
Provides-Extra: api-source
Requires-Dist: youtube-transcript-api>=1.0.0; extra == 'api-source'
Provides-Extra: dev
Requires-Dist: black>=24.0.0; extra == 'dev'
Requires-Dist: build>=1.0.0; extra == 'dev'
Requires-Dist: httpx>=0.24.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
Requires-Dist: pytest-mock>=3.10.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.8.0; extra == 'dev'
Requires-Dist: twine>=5.0.0; extra == 'dev'
Provides-Extra: extract
Requires-Dist: fpdf2>=2.7.0; extra == 'extract'
Requires-Dist: srt>=3.5.0; extra == 'extract'
Description-Content-Type: text/markdown

# subxx

**YouTube transcript / subtitle fetching toolkit for Python** - Download, extract, and process subtitles from video URLs with a simple CLI or HTTP API.

[![Version](https://img.shields.io/badge/version-0.5.0-blue.svg)](https://gist.github.com/cprima/1ec077cb315295e349ee61dccf13f6b2)
[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![License: CC BY 4.0](https://img.shields.io/badge/License-CC%20BY%204.0-lightgrey.svg)](https://creativecommons.org/licenses/by/4.0/)
[![Development Status](https://img.shields.io/badge/status-alpha-orange.svg)](https://github.com/cprima/subxx)

---

## Features

- **Download YouTube subtitles** from videos and channels (powered by [yt-dlp](https://github.com/yt-dlp/yt-dlp))
- **Multiple output formats**: SRT, VTT, TXT, Markdown, PDF
- **JSON output**: Machine-readable output with `--json` and `--json-file` flags
- **Importable module**: Use as a Python library with dict-based return values
- **Text extraction** with automatic subtitle cleanup and optional timestamp markers
- **Language selection**: Download specific languages or all available subtitles
- **Batch processing**: Process multiple URLs from a file
- **Configuration files**: Project and global settings via TOML
- **HTTP API**: Optional FastAPI server for programmatic access
- **Dry-run mode**: Preview operations without downloading
- **Filename sanitization**: Safe, nospace, or slugify modes

---

## Table of Contents

- [Installation](#installation)
- [Quick Start](#quick-start)
- [Module Usage (Python Library)](#module-usage-python-library)
- [Usage](#usage)
  - [List Available Subtitles](#list-available-subtitles)
  - [Download Subtitles](#download-subtitles)
  - [JSON Output](#json-output)
  - [Text Extraction](#text-extraction)
  - [Batch Processing](#batch-processing)
  - [Extract from Files](#extract-from-files)
- [Configuration](#configuration)
- [Makefile Shortcuts](#makefile-shortcuts)
- [HTTP API](#http-api)
- [Development](#development)
- [Testing](#testing)
- [License](#license)

---

## Installation

### Requirements

- Python 3.9 or higher
- [uv](https://github.com/astral-sh/uv) package manager (recommended)

### Install with uv (recommended)

```bash
# Clone or download the project
git clone https://gist.github.com/cprima/subxx
cd subxx

# Install core dependencies
uv sync

# Install with optional features
uv sync --extra extract      # Text extraction (txt/md/pdf)
uv sync --extra api          # HTTP API server
uv sync --extra dev          # Development tools (pytest)

# Install all features
uv sync --extra extract --extra api --extra dev
```

### Using Make (Windows)

```bash
make install          # Core dependencies
make install-all      # All dependencies (extract + api + dev)
```

---

## Quick Start

### Basic Usage

```bash
# List available subtitles
uv run subxx list https://youtu.be/VIDEO_ID

# Download English subtitle (SRT format, default)
uv run subxx subs https://youtu.be/VIDEO_ID

# Extract to plain text
uv run subxx subs https://youtu.be/VIDEO_ID --txt

# Extract to Markdown with 5-minute timestamps
uv run subxx subs https://youtu.be/VIDEO_ID --md -t 300

# Extract to PDF
uv run subxx subs https://youtu.be/VIDEO_ID --pdf

# Get JSON output for automation
uv run subxx list https://youtu.be/VIDEO_ID --json
uv run subxx subs https://youtu.be/VIDEO_ID --json-file output.json
```

### With Makefile

```bash
# Quick Markdown extraction (just paste video ID)
make md VIDEO_ID=dQw4w9WgXcQ

# With timestamps
make md VIDEO_ID=dQw4w9WgXcQ TIMESTAMPS=300
```

---

## Module Usage (Python Library)

subxx can be imported and used as a Python library. Core functions return typed
[pydantic](https://docs.pydantic.dev) result models (v0.5.0+; v0.4.x returned dicts, see
[Migrating to 0.5.0](#migrating-to-050)).

### Installation

```bash
# From test.pypi
pip install -i https://test.pypi.org/simple/ subxx==0.4.1

# Or with uv
uv add subxx==0.4.1 --index https://test.pypi.org/simple/
```

### Basic Example

```python
from subxx import fetch_subs, extract_text

# Download subtitles
result = fetch_subs(
    url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
    langs="en",
    fmt="srt",
    output_dir="./subs",
    logger=None  # Silent mode
)

if result.status == "success":
    print(f"Downloaded: {result.video.title}")
    for f in result.files:
        print(f"  {f.language}: {f.path}")
else:
    print(f"Error {result.error.code.value}: {result.error.message}")
```

### Result Format

Every function and every CLI command (`--json`) reports through one of four models sharing an envelope:
`SubsResult` (`fetch_subs`, `list`), `VideosResult` (`list_videos`), `ThumbnailResult` (`fetch_thumbnail`),
`ExtractionResult` (`extract_text`). `subxx schema <subs|videos|thumbnail|extraction>` prints the JSON Schema.

```json
{
  "schema_version": "1",
  "kind": "subs",
  "status": "success",
  "url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
  "video": {"id": "dQw4w9WgXcQ", "title": "Rick Astley - Never Gonna Give You Up", "duration": 213.0},
  "files": [{"path": "subs/video.en.srt", "format": "srt", "size_bytes": 4127, "language": "en", "auto_generated": false}],
  "languages": [{"code": "en", "name": "en", "auto": false}],
  "download_info": {"requested_languages": "en", "format": "srt", "auto_generated_fallback": true,
                    "output_directory": "subs", "downloaded_at": "2026-09-26T10:00:00Z", "dry_run": false},
  "attempts": []
}
```

- `status` is `success`, `skipped` or `error`. On error, `error` is `{"code", "message", "http_status", "retry_after_seconds"}`.
- `result.exit_code` (and the CLI exit code) comes from the error code.
- `attempts` lists automatic retries (see [Automatic Retry](#automatic-retry)); the full yt-dlp metadata is only
  included as `metadata` with `include_metadata=True` / `--full`: a `VideoInfo` with typed fields (title, duration,
  chapters, channel, ...) plus every other key yt-dlp returned. `None` fields are omitted from JSON.

### Error Codes

| Code | Meaning | Retryable | Exit code |
|------|---------|-----------|-----------|
| `RATE_LIMITED` | HTTP 429; wait and rerun | yes | 7 |
| `NETWORK_ERROR` | timeout, 5xx, connection problems | yes | 3 |
| `BLOCKED` | HTTP 403 or a bot check | no | 3 |
| `NOT_FOUND` | removed, private or unavailable | no | 3 |
| `NO_SUBTITLES` | no subtitles for the requested languages | no | 2 |
| `NO_THUMBNAIL`, `FILE_EXISTS`, `FILE_NOT_FOUND`, `FILE_ERROR`, `WRITE_ERROR`, `FORMAT_NOT_SUPPORTED`, `INVALID_FORMAT` | file and format problems | no | 6 (`NO_THUMBNAIL`: 3) |
| `MISSING_DEPENDENCY`, `UNSUPPORTED` | optional package missing, or the source lacks the capability | no | 5 |

### Complete Example

```python
from subxx import fetch_subs, extract_text

# 1. Download subtitle
result = fetch_subs(
    url="https://youtube.com/watch?v=...",
    langs="en",
    fmt="srt",
    auto=True,
    output_dir="./transcripts",
    logger=None
)

if result.status != "success":
    print(f"Error: {result.error.message}")
    raise SystemExit(result.exit_code)

# 2. Extract to markdown
subtitle_file = result.files[0].path
extract_result = extract_text(
    subtitle_file=subtitle_file,
    output_format="md",
    use_chapters=True,
    logger=None
)

if extract_result.status == "success":
    print(f"Extracted to: {extract_result.output_files[0].path}")
```

### Available Functions

```python
from subxx import (
    fetch_subs,        # Download subtitles → SubsResult
    fetch_thumbnail,   # Download the thumbnail → ThumbnailResult
    list_videos,       # List a channel/playlist → VideosResult
    extract_text,      # Extract text from srt → ExtractionResult
    get_source,        # Build a source (yt-dlp, retry) from config
    load_config,       # Load .subxx.toml config → dict
    get_default,       # Get config default value
    setup_logging,     # Configure logging
    ErrorCode,         # error codes with .exit_code and .retryable
)
```

### Migrating to 0.5.0

Results are models, not dicts (breaking). Access fields as attributes:

| 0.4.x | 0.5.0 |
|-------|-------|
| `result["status"]` | `result.status` |
| `result["video_id"]`, `result["video_title"]` | `result.video.id`, `result.video.title` |
| `result["files"][0]["path"]` | `result.files[0].path` |
| `result["available_languages"]`, `["auto"]` | `result.languages`, `.auto` |
| `result["error"]`, `result["error_code"]` | `result.error.message`, `result.error.code` |
| `result["metadata"]` (a yt-dlp dict) | `result.metadata` (a `VideoInfo`, only with `include_metadata=True`; `.model_dump()` for the dict) |
| `source.info(url)["title"]` (custom sources) | `source.info(url)` returns a `VideoInfo`: `.title`, `.subtitles`, `.chapters`, ... |
| `list_videos()` entries `{"id", "title", "url"}` | `Video` objects: `.id`, `.title`, `.url` |
| `extract_text(...)["output_files"][0]["path"]` | `extract_text(...).output_files[0].path` |

`result.model_dump()` gives a plain dict. JSON output changed shape accordingly (see [Result Format](#result-format)).

---

## Usage

### List Available Subtitles

Preview available subtitle languages without downloading:

```bash
# Traditional output
uv run subxx list https://youtu.be/VIDEO_ID

# JSON output
uv run subxx list https://youtu.be/VIDEO_ID --json

# Save to file
uv run subxx list https://youtu.be/VIDEO_ID --json-file metadata.json
```

**Output:**
```
📹 Video: Example Video Title
🕒 Duration: 12:34

✅ Manual subtitles:
   - en
   - es

🤖 Auto-generated subtitles:
   - en, de, fr, ja, ko, pt, ru, zh-Hans, ...
```

**Options:**
- `-v, --verbose` - Debug output
- `-q, --quiet` - Errors only

---

### Download Subtitles

#### Format Selection

Download subtitle files in SRT or VTT format:

```bash
# Download SRT (default)
uv run subxx subs https://youtu.be/VIDEO_ID

# Download VTT
uv run subxx subs https://youtu.be/VIDEO_ID --vtt

# Using --fmt flag
uv run subxx subs https://youtu.be/VIDEO_ID -f srt
```

**Behavior**: Subtitle files (SRT/VTT) are downloaded and kept on disk.

#### Language Selection

```bash
# Download English (default)
uv run subxx subs https://youtu.be/VIDEO_ID

# Download specific language
uv run subxx subs https://youtu.be/VIDEO_ID -l de

# Download multiple languages
uv run subxx subs https://youtu.be/VIDEO_ID -l "en,de,fr"

# Download all available languages
uv run subxx subs https://youtu.be/VIDEO_ID -l all
```

#### Output Directory

```bash
# Save to specific directory
uv run python __main__.py subs https://youtu.be/VIDEO_ID -o ~/Downloads/subs

# Use current directory (default)
uv run python __main__.py subs https://youtu.be/VIDEO_ID -o .
```

#### Filename Sanitization

```bash
# Safe mode: Remove unsafe characters, keep spaces (default)
uv run python __main__.py subs URL --sanitize safe

# No spaces: Replace spaces with underscores
uv run python __main__.py subs URL --sanitize nospaces

# Slugify: Lowercase, hyphens, URL-safe
uv run python __main__.py subs URL --sanitize slugify
```

**Examples:**
- `safe`: `"My Video Title.srt"` → `"My Video Title.srt"`
- `nospaces`: `"My Video Title.srt"` → `"My_Video_Title.srt"`
- `slugify`: `"My Video Title.srt"` → `"my-video-title.srt"`

#### Overwrite Handling

```bash
# Prompt before overwriting (default)
uv run python __main__.py subs URL

# Force overwrite without prompting
uv run python __main__.py subs URL --force

# Skip existing files
uv run python __main__.py subs URL --skip-existing
```

#### Auto-Generated Subtitles

```bash
# Include auto-generated subtitles (default)
uv run python __main__.py subs URL --auto

# Only manual subtitles
uv run python __main__.py subs URL --no-auto
```

#### Dry Run

Preview what would be downloaded without actually downloading:

```bash
uv run python __main__.py subs URL --dry-run
```

**Output:**
```
[DRY RUN] Would download subtitle: en
```

---

### JSON Output

Machine-readable JSON output for automation and scripting. Every result has the same envelope
(`schema_version`, `kind`, `status`, `error`, `attempts`); see [Result Format](#result-format), and
`subxx schema <kind>` for the JSON Schema.

#### Available Commands with JSON Support

- `list` - List available languages
- `subs` - Download subtitles (`--full` adds the complete video metadata as `metadata`)
- `videos` - List a channel's videos

#### Output to stdout

```bash
# List command with JSON
uv run subxx list "https://youtu.be/dQw4w9WgXcQ" --json

# Subs command with JSON
uv run subxx subs "https://youtu.be/dQw4w9WgXcQ" --json
```

**Example JSON output**:
```json
{
  "schema_version": "1",
  "kind": "subs",
  "status": "success",
  "url": "https://youtu.be/dQw4w9WgXcQ",
  "video": {"id": "dQw4w9WgXcQ", "title": "Rick Astley - Never Gonna Give You Up..."},
  "files": [
    {
      "path": "Rick Astley - Never Gonna Give You Up.dQw4w9WgXcQ.NA.en.srt",
      "format": "srt",
      "language": "en",
      "auto_generated": false
    }
  ],
  "languages": [
    {"code": "en", "name": "en", "auto": false}
  ],
  "attempts": []
}
```

#### Save to file

```bash
# Save JSON to file
uv run subxx list URL --json-file metadata.json
uv run subxx subs URL --json-file result.json

# Both stdout and file
uv run subxx subs URL --json --json-file result.json
```

#### Use in Scripts

```bash
#!/bin/bash

# Get video metadata
metadata=$(uv run subxx list "$VIDEO_URL" --json)
video_title=$(echo "$metadata" | jq -r '.video.title')

echo "Downloading: $video_title"

# Download with JSON output
uv run subxx subs "$VIDEO_URL" --json-file download.json

# Check if successful
if [ "$(jq -r '.status' download.json)" == "success" ]; then
    echo "Success! Downloaded $(jq -r '.files | length' download.json) files"
fi
```

---

### Text Extraction

Extract clean, readable text from subtitles by automatically removing timestamps and formatting.

**Key behavior**: When using text formats (txt/md/pdf), subxx:
1. Downloads the subtitle as SRT
2. Extracts the text content
3. **Automatically deletes the SRT file**

#### Plain Text

```bash
# Extract to plain text
uv run python __main__.py subs URL --txt
```

**Output**: `Video_Title.VIDEO_ID.en.txt`

**Example content:**
```
Hello world.
This is a subtitle.
Welcome to the video.
```

#### Markdown

```bash
# Extract to Markdown
uv run python __main__.py subs URL --md

# Markdown with timestamp markers every 5 minutes
uv run python __main__.py subs URL --md -t 300

# Markdown with timestamp markers every 30 seconds
uv run python __main__.py subs URL --md -t 30
```

**Output**: `Video_Title.VIDEO_ID.en.md`

**Example content (with timestamps):**
```markdown
## [0:00]

Hello world.
This is a subtitle.

## [5:00]

Welcome to the next section.
More content here.

## [10:00]

Final section of the video.
```

#### PDF

```bash
# Extract to PDF
uv run python __main__.py subs URL --pdf

# PDF with timestamp markers
uv run python __main__.py subs URL --pdf -t 300
```

**Output**: `Video_Title.VIDEO_ID.en.pdf`

**Requirements**: Install extraction dependencies:
```bash
uv sync --extra extract
```

#### Timestamp Intervals

Add timestamp markers at regular intervals for long-form content:

```bash
# Every 5 minutes (300 seconds)
uv run python __main__.py subs URL --md -t 300

# Every 30 seconds
uv run python __main__.py subs URL --txt -t 30

# Every 10 minutes
uv run python __main__.py subs URL --pdf -t 600
```

**Format**: Timestamps appear as `## [0:00]`, `## [5:00]`, `## [10:00]`, etc.

---

### Batch Processing

Download subtitles for multiple URLs from a file:

```bash
# Create URLs file (one URL per line)
cat > urls.txt << EOF
https://youtu.be/VIDEO_ID_1
https://youtu.be/VIDEO_ID_2
# This is a comment
https://youtu.be/VIDEO_ID_3
EOF

# Process all URLs
uv run python __main__.py batch urls.txt

# With options
uv run python __main__.py batch urls.txt -l "en,de" -f srt -o ~/subs
```

**Options:**
- `-l, --langs` - Language codes (default: en)
- `-f, --fmt` - Output format (default: srt)
- `-o, --output-dir` - Output directory (default: .)
- `--sanitize` - Filename sanitization mode (default: safe)
- `-v, --verbose` - Verbose output
- `-q, --quiet` - Quiet mode

**URL File Format** (yt-dlp standard):
- One URL per line
- Lines starting with `#` are comments
- Empty lines are ignored

---

### Extract from Files

Extract text from existing subtitle files:

```bash
# Extract SRT to plain text
uv run python __main__.py extract video.srt

# Extract to Markdown
uv run python __main__.py extract video.srt -f md

# Extract to PDF
uv run python __main__.py extract video.srt -f pdf

# With timestamp markers every 5 minutes
uv run python __main__.py extract video.srt -f md -t 300

# Specify output file
uv run python __main__.py extract video.srt -o output.txt

# Force overwrite
uv run python __main__.py extract video.srt --force
```

**Supported input formats**: SRT, VTT

---

## Configuration

### Config File Locations

Configuration files are loaded in priority order:

1. `./.subxx.toml` (project-specific, current directory)
2. `~/.subxx.toml` (user global, home directory)

### Priority Chain

Settings are resolved in this order (highest to lowest):

1. **CLI flags** (e.g., `--langs en`, `--fmt srt`)
2. **Config file** (`.subxx.toml`)
3. **Hardcoded defaults**

### Example Configuration

Copy `.subxx.toml.example` to `.subxx.toml` or `~/.subxx.toml`:

```bash
cp .subxx.toml.example ~/.subxx.toml
```

**Example config:**

```toml
[defaults]
# Language codes (comma-separated or "all")
langs = "en"

# Output format: srt, vtt, txt, md, pdf
fmt = "md"

# Include auto-generated subtitles
auto = true

# Output directory (supports ~)
output_dir = "~/Downloads/subtitles"

# Filename sanitization: safe, nospaces, slugify
sanitize = "safe"

# Timestamp interval (seconds) for txt/md/pdf
timestamps = 300  # 5-minute intervals

[logging]
# Log level: DEBUG, INFO, WARNING, ERROR
level = "INFO"

# Log file (optional)
log_file = "~/.subxx/subxx.log"
```

### Subtitle Sources and yt-dlp Options

Subtitles come from a pluggable *source* (`sources.py`). The default is yt-dlp; choose another with
`[source] backend` or `--source`:

| Source | Extra | Subtitles | List channel videos | Thumbnails | Notes |
|--------|-------|-----------|---------------------|------------|-------|
| `ytdlp` (default) | none | yes | yes | yes | Full metadata, chapters, channel info |
| `transcript-api` | `subxx[api-source]` | yes | no (`UNSUPPORTED`) | no (`UNSUPPORTED`) | Uses `youtube-transcript-api`; text only |

List a channel's videos and download thumbnails (yt-dlp source):

```bash
# use a tab URL such as .../videos
uv run subxx videos https://www.youtube.com/channel/CHANNEL_ID/videos --limit 10
uv run subxx videos https://www.youtube.com/channel/CHANNEL_ID/videos --json

# thumbnail next to the subtitles: jpg/png need ffmpeg, "native" keeps the original (webp)
uv run subxx subs <url> --thumbnail jpg      # or [defaults] thumbnail = "jpg"
```

```python
from subxx import list_videos, fetch_thumbnail, get_source, load_config

source = get_source(load_config())
listing = list_videos("https://www.youtube.com/channel/CHANNEL_ID/videos", limit=10, source=source)
# VideosResult: listing.status, listing.videos[i].id/.title/.url, listing.error
thumb = fetch_thumbnail(listing.videos[0].url, fmt="jpg", output_dir="out",
                        out_template="thumbnail", skip_existing=True, source=source)
```

yt-dlp options come from the `[ytdlp]` table and are passed to every yt-dlp request:

```toml
[ytdlp]
sleep_interval = 20        # random pause between requests: 20-60 s
max_sleep_interval = 60
sleep_interval_subtitles = 5
retries = 5
socket_timeout = 30
```

Options subxx derives from its own arguments (`writesubtitles`, `subtitleslangs`, `outtmpl`, ...) are
ignored with a warning; cookie options (`cookiefile`, `cookiesfrombrowser`) are not supported.

In Python, build a source from the same config and pass it in:

```python
from subxx import fetch_subs, load_config, get_source

source = get_source(load_config())          # or get_source({}, "transcript-api")
result = fetch_subs(url, langs="en", source=source)
```

Any object with `info(url)` and `download(...)` (see `SubtitleSource` in `sources.py`) can be used.

### Automatic Retry

Retrying is **opt-in**: without a `[retry]` table nothing is repeated, and every value must be set (there is no
built-in schedule).

```toml
[retry]
max_attempts = 4        # total tries per operation, the first one included
base_delay = 30         # seconds; doubles after each failure, with jitter (half to full backoff)
max_delay = 600         # cap for one wait
max_total_wait = 1200   # optional: cap for all waits of one operation
```

- Only retryable errors are repeated (`RATE_LIMITED`, `NETWORK_ERROR`); `Retry-After` from the server is honored,
  and if it exceeds `max_delay` subxx gives up instead of waiting.
- Every retry is recorded in `result.attempts`; the CLI logs `RATE_LIMITED, HTTP 429: retrying in 63s (attempt 2/4)`.
- `--retries N` overrides `max_attempts` on `subs`, `list`, `videos` and `batch` (the other values still come from `[retry]`).
- This is a second layer on top of yt-dlp's own per-request `[ytdlp]` retries and sleeps, which multiply with it;
  `max_total_wait` bounds the worst case.
- After the retries are used up, a 429 exits with code **7**, and `batch` stops instead of sending more requests.
- Library: `get_source(config)` wraps the source when `[retry]` is present, or use `retry.RetryingSource(source, RetryPolicy(...))`.

### Use Case Configurations

**Configuration 1: Download SRT files to dedicated directory**
```toml
[defaults]
langs = "en"
fmt = "srt"
output_dir = "~/Downloads/subtitles"
```

**Configuration 2: Auto-extract to Markdown with timestamps**
```toml
[defaults]
langs = "en"
fmt = "md"
timestamps = 300
output_dir = "~/Documents/transcripts"
```

**Configuration 3: Multiple languages, plain text**
```toml
[defaults]
langs = "en,de,fr"
fmt = "txt"
sanitize = "slugify"
output_dir = "./subtitles"
```

---

## Makefile Shortcuts

### Available Targets

```bash
# Installation
make install          # Core dependencies
make install-all      # All dependencies (extract + api + dev)

# Testing
make test             # Run all tests
make test-unit        # Unit tests only
make test-integration # Integration tests only
make test-coverage    # Tests with coverage report

# Usage
make list VIDEO_URL=https://youtu.be/VIDEO_ID
make subs VIDEO_URL=https://youtu.be/VIDEO_ID
make md VIDEO_ID=VIDEO_ID                       # Quick Markdown extraction
make md VIDEO_ID=VIDEO_ID TIMESTAMPS=300        # With timestamps

# Utilities
make version          # Show version
make clean            # Clean cache files
make clean-all        # Clean everything including .venv
```

### Examples

```bash
# Quick Markdown extraction (just paste video ID)
make md VIDEO_ID=dQw4w9WgXcQ

# With 5-minute timestamps
make md VIDEO_ID=lHuxDMMkGJ8 TIMESTAMPS=300

# List subtitles
make list VIDEO_URL=https://youtu.be/dQw4w9WgXcQ

# Download with languages
make subs VIDEO_URL=https://youtu.be/dQw4w9WgXcQ LANGS=en,de
```

---

## HTTP API

Start an HTTP API server for programmatic access (requires API dependencies):

### Installation

```bash
# Install API dependencies
uv sync --extra api

# Or with Make
make install-api
```

### Start Server

```bash
# Start on localhost:8000 (default)
uv run python __main__.py serve

# Custom host/port
uv run python __main__.py serve --host 127.0.0.1 --port 8080
```

**Security Warning**: The API has NO authentication and should ONLY run on localhost (127.0.0.1).

### API Endpoints

#### POST /subs

Fetch subtitles and return content directly.

**Request:**
```json
{
  "url": "https://youtu.be/VIDEO_ID",
  "langs": "en",
  "fmt": "srt",
  "auto": true,
  "sanitize": "safe"
}
```

**Response:** Subtitle file content as plain text.

**Example:**
```bash
curl -X POST http://127.0.0.1:8000/subs \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://youtu.be/dQw4w9WgXcQ",
    "langs": "en",
    "fmt": "srt"
  }'
```

Errors map from the error code: `NO_SUBTITLES`/`NOT_FOUND` → 404, `RATE_LIMITED` → 429, anything else → 500.

#### POST /subs/result

Same request, but the response is the typed `SubsResult` (see [Result Format](#result-format)) as JSON, with the
HTTP status mapped from the error code as above. The download is temporary, so `files[].path` holds file names only.

```bash
curl -X POST http://127.0.0.1:8000/subs/result -H "Content-Type: application/json" \
  -d '{"url": "https://youtu.be/dQw4w9WgXcQ", "langs": "en"}'
```

The response schema is in the OpenAPI docs at `/docs` and via `subxx schema subs`.

#### GET /health

Health check endpoint.

**Response:**
```json
{
  "status": "ok",
  "service": "subxx"
}
```

### API Documentation

Interactive API docs available at:
- Swagger UI: `http://127.0.0.1:8000/docs`
- ReDoc: `http://127.0.0.1:8000/redoc`

---

## Development

### Setup Development Environment

```bash
# Clone repository
git clone https://gist.github.com/cprima/subxx
cd subxx

# Install all dependencies (core + extract + api + dev)
uv sync --extra extract --extra api --extra dev

# Or with Make
make install-all
```

### Project Structure

**Updated in v0.4.1** - Restructured for Python best practices:

```
subxx/
├── subxx.py                 # Core library functions (returns dicts)
├── cli.py                   # CLI + API implementation (Typer/FastAPI)
├── __main__.py              # Minimal entry point (3 lines)
├── test_subxx.py            # Test suite (pytest)
├── conftest.py              # Pytest configuration
├── pyproject.toml           # Project metadata and dependencies
├── Makefile                 # Build and test automation
├── .subxx.toml.example      # Example configuration file
└── !README.md               # This file
```

### Key Components

- **`subxx.py`**: Core library (library-first design)
  - `fetch_subs()` → dict - Download subtitles, return structured data
  - `extract_text()` → dict - Extract text from subtitles, return structured data
  - `load_config()` → dict - Configuration management
  - Helper functions for parsing, sanitization, logging
  - **Importable as Python module**

- **`cli.py`**: CLI + API implementation
  - Typer commands: `list`, `subs`, `batch`, `extract`, `serve`, `version`
  - FastAPI HTTP server
  - JSON output handling (`--json`, `--json-file`)
  - Traditional console output with emojis

- **`__main__.py`**: Minimal entry point (Python best practice)
  - 3 lines: import and run CLI
  - Enables `python -m subxx` usage

---

## Testing

### Run Tests

```bash
# All tests
make test

# Unit tests only (fast, no network)
make test-unit

# Integration tests only
make test-integration

# With coverage report
make test-coverage

# Verbose output
make test-verbose
```

### Test Categories

- **Unit tests** (`@pytest.mark.unit`): No external dependencies, mocked I/O
- **Integration tests** (`@pytest.mark.integration`): May use files/network
- **E2E tests** (`@pytest.mark.e2e`): Real YouTube API, requires internet
- **Slow tests** (`@pytest.mark.slow`): Network I/O, real downloads

### Running Specific Test Categories

```bash
# Run all tests except e2e (fast, for CI)
pytest -m "not e2e"

# Run only e2e tests (slow, requires internet)
pytest -m e2e

# Run unit tests only
pytest -m unit
```

### Test Coverage

Current coverage: **~50 tests** (unit, integration, and e2e)

Key areas tested:
- Configuration loading and defaults
- Language parsing
- Filename sanitization
- Text extraction (txt/md/pdf)
- Timestamp markers
- CLI commands
- Overwrite protection
- Real YouTube subtitle download (e2e)

---

## Exit Codes

- `0` - Success
- `1` - User cancelled
- `2` - No subtitles available
- `3` - Network error
- `4` - Invalid URL
- `5` - Configuration error
- `6` - File error

---

## Troubleshooting

### Missing Dependencies for Text Extraction

**Error:**
```
❌ Error: Missing dependencies for text extraction
```

**Solution:**
```bash
uv sync --extra extract
```

### Missing Dependencies for API

**Error:**
```
❌ Error: API dependencies not installed
```

**Solution:**
```bash
uv sync --extra api
```

### Windows Console Encoding Issues

If you see encoding errors on Windows, the tool automatically attempts to reconfigure stdout/stderr to UTF-8. If issues persist, use:

```bash
# Set console to UTF-8
chcp 65001
```

### yt-dlp Network Errors

If downloads fail with network errors:

1. Update yt-dlp:
   ```bash
   uv sync --upgrade
   ```

2. Check firewall/proxy settings

3. Try with `--verbose` for debug output:
   ```bash
   uv run python __main__.py subs URL --verbose
   ```

---

## Roadmap

### Completed (v0.4.x)
- [x] **JSON output support** (`--json`, `--json-file`)
- [x] **Importable Python module** (library-first architecture)
- [x] **Published package on test.pypi.org**
- [x] **Pythonic project structure** (cli.py, minimal __main__.py)

### Future Enhancements
- [ ] Publish to PyPI (production)
- [ ] Progress bars for downloads
- [ ] Retry logic for network failures
- [ ] Subtitle merging/combining
- [ ] Translation support
- [ ] Docker container
- [ ] GitHub Actions CI/CD
- [ ] SRT/VTT format conversion
- [ ] Subtitle editing/manipulation
- [ ] Batch command JSON support
- [ ] Extract command JSON support

---

## Contributing

Contributions welcome! This is an alpha project under active development.

### How to Contribute

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass: `make test`
6. Submit a pull request

### Guidelines

- Follow existing code style
- Add docstrings for new functions
- Update tests for changes
- Update README for new features
- Keep commits focused and atomic

---

## License

This project is licensed under **CC BY 4.0** (Creative Commons Attribution 4.0 International).

You are free to:
- **Share** - Copy and redistribute the material
- **Adapt** - Remix, transform, and build upon the material

Under the following terms:
- **Attribution** - You must give appropriate credit

See [LICENSE](https://creativecommons.org/licenses/by/4.0/) for full details.

---

## Credits

- Built with [yt-dlp](https://github.com/yt-dlp/yt-dlp) for video subtitle extraction
- CLI powered by [Typer](https://typer.tiangolo.com/)
- API built with [FastAPI](https://fastapi.tiangolo.com/)
- Text extraction using [srt](https://github.com/cdown/srt) and [fpdf2](https://github.com/py-pdf/fpdf2)

---

## Author

**Christian Prior-Mamulyan**
- Email: cprior@gmail.com
- GitHub: [@cprima](https://github.com/cprima)

---

## Support

- Report issues: [GitHub Issues](https://gist.github.com/cprima/subxx/issues)
- Documentation: [GitHub Gist](https://gist.github.com/cprima/subxx)

---

**subxx** - Simple, powerful YouTube transcript / subtitle fetching for Python.
