# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Automated, show-agnostic podcast/audio production pipeline using ElevenLabs TTS API. Turns a markdown production script into a podcast-ready MP3.

## Package Structure

The project is packaged as `xil-pipeline` (import name `xil_pipeline`) using hatchling. All pipeline and utility scripts live under `src/xil_pipeline/`:

```
src/xil_pipeline/          # Python package (24 modules)
  __init__.py              # version + key re-exports
  xil.py                   # Unified `xil` command dispatcher
  models.py                # Pydantic data models, slug/path resolution
  mix_common.py            # Shared mixing utilities
  sfx_common.py            # SFX library management, ID3 tagging
  timeline_viz.py          # Timeline visualization
  xil_init.py              # Project scaffolding (xil-init command, --type aware)
  XILP000_*.py … XILP012_*.py   # Pipeline stages
  XILU001_*.py … XILU009_*.py   # Utility scripts
tests/                     # Pytest test suite
docs/                      # MkDocs documentation
pyproject.toml             # Packaging config (hatchling)
project.json               # Show name config (runtime, read from CWD)
speakers.json              # Speaker definitions (optional, overrides built-in defaults)
cast_*.json, sfx_*.json    # Episode configs (workspace data, stays at root)
```

Install for development: `pip install -e ".[all,dev]"`

All internal imports use the package namespace: `from xil_pipeline.models import ...`

## Environment

- Python 3.12+, virtualenv at `venv/`
- WSL2 (Linux on Windows)
- Activate: `source venv/bin/activate`
- Install: `pip install -e ".[all,dev]"` (editable install with all optional deps)
- Core packages: `elevenlabs`, `pydub`, `pydantic`, `mutagen`, `httpx`
- Optional: `google-genai`, `gTTS`, `pyttsx3`, `ollama`
- ElevenLabs API key via `ELEVENLABS_API_KEY` env var
- Audio playback via `mpg123` in WSL

## Workspace Root (`XIL_PROJECTROOT`)

All pipeline commands resolve content paths relative to the **workspace root**. By default this is the current working directory (existing behaviour). Set `XIL_PROJECTROOT` to point at a content directory from anywhere:

```bash
export XIL_PROJECTROOT=/path/to/xil-content
xil produce --episode S04E02   # works from any directory
xil-gui                        # GUI shows correct workspace
```

Resolution order for `get_workspace_root()` (in `models.py`):

1. `XIL_PROJECTROOT` environment variable (absolute path, tilde-expanded)
2. `Path.cwd()` — current working directory (no env var set)

`xil-init` with no directory argument scaffolds into `XIL_PROJECTROOT` when set, otherwise into the current directory.

`logs/`, `configs/`, `parsed/`, `stems/`, `SFX/`, `daw/`, `masters/`, `cues/`, `scripts/`, `posts/` all resolve under the workspace root. This enables a clean separation of the installed software from user content — install `xil-pipeline` once via `pip`, point `XIL_PROJECTROOT` at a content-only directory.

## Project Configuration

`project.json` at the workspace root declares the show name and optional season title used across the pipeline:

```json
{
    "show": "THE 413",
    "season": 3,
    "season_title": "The Holiday Shift"
}
```

All scripts accept a `--show` CLI flag to override the show name. Resolution order: `--show` arg > `project.json` > hardcoded fallback `"sample"`.

The `season_title` key in `project.json` is the workspace-level default for the season/arc title. When a script header contains `Arc: "…"`, that value takes precedence; when absent, `project.json` `season_title` fills in. Resolution order: script header `Arc:` > `project.json` `season_title` > `None`. The `{season_title}` placeholder in PREAMBLE/POSTAMBLE script text resolves from this value.

The `season` key in `project.json` is the workspace-level default season number. When a script header contains `Season N:`, that value takes precedence; when absent, `project.json` `season` fills in. Resolution order: script header `Season N:` > `project.json` `season` > `None`.

### Content Type

`project.json` supports an optional `"type"` field (default: `"podcast"`):

```json
{
  "show": "THE 413",
  "type": "drama",
  "season": 3,
  "season_title": "The Architect"
}
```

Four types are supported: **podcast**, **audiobook**, **drama**, **special**. The type drives:
- Section map used by `xil-parse` (e.g. `CHAPTER ONE` for audiobook, `PROLOGUE/ACT ONE` for drama)
- Default `gap_ms` for `xil-assemble` / `xil-daw` (400 ms audiobook, 800 ms drama, 600 ms others)
- Sample script, speakers.json, and subdirectory layout generated by `xil-init --type`
- Audiobook type adds `"tag_format": "V{volume:02d}C{chapter:02d}"` to `project.json` for `V01C01`-style tags

`project.json` without a `type` field defaults to `"podcast"` with no change in behavior.

### Workspace Layout (0.1.8+)

New workspaces created with `xil-init` use a normalized directory layout:

```
configs/{slug}/speakers.json        ← was speakers.json at root
configs/{slug}/cast_{tag}.json      ← was cast_{slug}_{tag}.json at root
configs/{slug}/sfx_{tag}.json       ← was sfx_{slug}_{tag}.json at root
parsed/{slug}/parsed_{tag}.json     ← was parsed/parsed_{slug}_{tag}.json
daw/{slug}/{tag}/                   ← was daw/{tag}/
masters/{slug}/{tag}_master.mp3     ← was masters/{slug}_{tag}_master.mp3
cues/{slug}/cues_{tag}.md           ← was cues/cues_{slug}_{tag}.md
stems/{slug}/{tag}/                 ← unchanged
```

Existing pre-0.1.8 workspaces continue to work automatically — `derive_paths()` detects the legacy layout (cast config at root) and returns legacy paths. Run `xil migrate-workspace` to move files to the new layout.

File paths are derived dynamically via `derive_paths(slug, tag)`. The slug is the show name lowercased with all non-alphanumeric characters removed (e.g., `"THE 413"` → `"the413"`, `"Night Owls"` → `"nightowls"`).

## Project Scaffolding

`xil-init` scaffolds a new show workspace with sample content:

```bash
xil-init my-show --show "Night Owls"
xil-init my-show --show "Night Owls" --type drama
xil-init my-show --show "Night Owls" --type audiobook
```

The `--type` flag (choices: `podcast` [default], `audiobook`, `drama`, `special`) selects:
- A type-specific sample script with appropriate sections and cast
- A type-specific `speakers.json` (single narrator for audiobook, full cast for drama, etc.)
- `project.json` `"type"` field + `"tag_format"` for audiobook (`V01C01`)

Creates: `project.json`, `speakers.json`, `scripts/sample_{tag}.md`, and empty subdirectories in the normalized layout. The sample script exercises all parser features so the user can immediately run `xil-scan` and `xil-parse --dry-run`.

## Speaker Configuration

`configs/{slug}/speakers.json` defines the speaker names the parser recognizes (created by `xil-init`):

```json
[
    {"display": "ADAM", "key": "adam"},
    {"display": "MR. PATTERSON", "key": "mr_patterson"},
    {"display": "FILM AUDIO (MARGARET'S VOICE)", "key": "film_audio"}
]
```

Resolution order: `--speakers PATH` flag > `configs/{slug}/speakers.json` > `speakers.json` in CWD (legacy fallback) > built-in defaults. The list is auto-sorted longest-first for correct compound-name matching. Both `xil-scan` (XILP000) and `xil-parse` (XILP001) accept the `--speakers` flag.

## Pre-Flight Script Scanner

`XILP000_script_scanner.py` — Scans a raw markdown script and reports recognized/unrecognized speakers and sections **before** running XILP001. Use this whenever onboarding a new script to catch missing speakers or `SECTION_MAP` entries early.

```bash
xil scan "scripts/<script>.md"
xil scan "scripts/<script>.md" --json
```

- No `--episode` flag required — reads only the script file, no side effects
- Exit code 0 = all recognized (safe to run XILP001); exit code 1 = action needed
- Imports XILP001's pure functions directly — no duplicated logic
- `--json` outputs machine-readable scan results
- `--speakers PATH` overrides the speaker list (see Speaker Configuration)

## Architecture: Nine-Stage Pipeline (+ Cues Ingester Pre-Processing)

### Stage 1: Script Parsing

`XILP001_script_parser.py` — Parses markdown production scripts into structured JSON.

```bash
xil parse "scripts/<script>.md" --episode S01E01 --preview 10
```

- Input: Markdown scripts in `scripts/` — supports both plain text (S01E01) and markdown-formatted (S01E02+) scripts transparently
- Two-pass normalization: `strip_markdown_escapes()` removes `\[`, `\]`, etc.; `strip_markdown_formatting()` removes `**`, `##`/`###` headings, trailing double-space line breaks
- Handles both single-line dialogue (`SPEAKER (dir) Text`) and multi-line dialogue (speaker, direction, text on separate lines) via pending-speaker state machine
- Standalone parenthetical acting notes like `(beat)` or `(pause)` within dialogue continuations are filtered from spoken text
- Square-bracket stage directions with unrecognized `direction_type` (acting notes like `[drawn out]`, `[quietly]`) are silently skipped rather than emitted as `type: direction, direction_type: None` noise entries
- Dividers: accepts both `===` (plain text) and `---` (markdown horizontal rules)
- End markers: stops at `END OF EPISODE` or `END OF PRODUCTION SCRIPT`
- Output: `parsed/parsed_<slug>_S01E01.json` — entries with seq, type, section, scene, speaker, direction, text, direction_type
- Output path derived from script header metadata (season/episode); override with `--output`
- `--episode S01E01` (optional) validates that the script header matches the intended episode tag
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- When `--episode` is provided and `cast_<slug>_S01E01.json` / `sfx_<slug>_S01E01.json` don't exist, auto-generates skeleton configs with `voice_id=TBD` and default SFX prompts; the cast skeleton includes `season_title` populated from the script header's `Arc: "…"` declaration (or `null` when absent)
- `season_title` is extracted from the `Arc: "…"` token in the script header (e.g. `THE 413 Season 1: Episode 1: "The Empty Booth" Arc: "The Holiday Shift"`) and stored in the parsed JSON; it is available as `{season_title}` in PREAMBLE/POSTAMBLE script text
- Supports `--quiet` (JSON only, skip summary) and `--debug` (write diagnostic CSV alongside JSON)
- Auto-generates BEAT variants (`BEAT — 3 SECONDS` etc.) as `type: "silence"` with duration parsed from the text (e.g. 3.0s)
- Auto-generates `AMBIENCE: STOP` and `AMBIENCE: * FADES OUT` directives as `type: "silence", duration_seconds: 0.0` stop markers — no audio asset needed
- Known speakers loaded from `speakers.json` (see Speaker Configuration); built-in defaults used as fallback
- `--speakers PATH` overrides the speaker list (see Speaker Configuration)
- Sections: COLD OPEN, OPENING CREDITS, ACT ONE, ACT TWO, MID-EPISODE BREAK, CLOSING

### Stage 1.5: Cues Sheet Ingestion (Pre-processing)

`XILP006_cues_ingester.py` — Parses a sound cues & music prompts markdown file into a structured asset manifest, audits the shared SFX library, and optionally enriches the episode sfx config or generates new assets.

```bash
xil cues --episode S02E03 --cues "cues/<file>.md"
xil cues --episode S02E03 --cues "cues/<file>.md" --enrich-sfx-config
xil cues --episode S02E03 --cues "cues/<file>.md" --generate
xil cues --episode S02E03 --cues "cues/<file>.md" --generate --enrich-sfx-config
```

- `--episode` or `--tag` (one required) derives the sfx config path (`sfx_<slug>_S02E03.json`)
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- `--cues PATH` explicit path to the cues markdown file; auto-detected from `cues/` if omitted and exactly one `.md` exists there (canonical name: `cues/cues_<slug>_S02E03.md`)
- Always writes `cues/cues_manifest_<TAG>.json` — structured JSON catalog of all parsed assets
- Always prints an audit report: EXISTS / REUSE / NEW status per asset, credit estimate for NEW generation
- `--enrich-sfx-config` — updates `sfx_<slug>_<TAG>.json` entries that reference a cues-sheet asset ID: replaces stub prompts with the full cues-sheet prompt and corrects duration (capped at 30s API limit)
- `--generate` — calls ElevenLabs Sound Effects API to generate NEW assets into `SFX/<asset-id>.mp3` (e.g. `SFX/sfx-boots-stamp-01.mp3`); skips assets already on disk; REUSE assets are never generated here
- `--dry-run` — suppresses API calls and sfx config writes; shows enrichment diff and generation credit estimate
- Parses three cue sheet sections: MUSIC CUES (heading blocks), AMBIENCE (heading blocks), SOUND EFFECTS (markdown tables per scene)
- Duration cap: assets longer than 30s are generated at 30s and flagged `[CAPPED]` in the audit

### Stage 2: Voice Generation

`XILP002_producer.py` — Calls ElevenLabs API to generate voice stems.

```bash
xil produce --episode S01E01 --dry-run
```

- `--episode` or `--tag` (one required) derives `cast_<slug>_S01E01.json` and `sfx_<slug>_S01E01.json`
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- Reads: parsed JSON + cast config; always loads SFX config (for INTRO/OUTRO MUSIC source lookup)
- Outputs: `stems/<slug>/<TAG>/{seq:03d}_{section}[-{scene}]_{speaker}.mp3` (e.g. `stems/the413/S01E01/003_cold-open_adam.mp3`)
- **Preamble/Postamble**: `PREAMBLE` and `POSTAMBLE` are first-class script sections parsed by XILP001 and written into the parsed JSON with contiguous seq numbers. XILP002 generates their voice stems through the standard `generate_voices()` loop, applying a per-section speed override from the cast config `preamble.speed` / `postamble.speed` field. No separate injection step — preamble/postamble entries exist in the parsed JSON from the start
- INTRO/OUTRO MUSIC source stems are copied from `sfx_config.effects["INTRO MUSIC"/"OUTRO MUSIC"].source` at produce time
- Supports `--start-from N` for resuming interrupted runs; `--stop-at N` to halt after a specific seq (useful for previewing a section without regenerating the full episode)
- Supports `--dry-run` to preview lines and TTS character cost without API calls; includes a per-speaker breakdown table (lines + chars to generate vs. already on disk) sorted by chars descending; per-entry marker: `[ ]` = will generate, `[=]` = stem exists/skip, `[x]` = out of range
- Supports `--terse` to truncate each line to 3 words (minimizes TTS character cost)
- Supports `--gen-sfx`, `--gen-music`, `--gen-ambience` to generate only the specified categories of stems (replaces deprecated `--sfx-music` which is kept as a shorthand for all three)
- Supports `--local-only` (used with `--gen-sfx`/`--gen-music`/`--gen-ambience`) to skip any effect that would require an API call — only assets already in `SFX/` (CACHED) or silence entries are placed; no credits spent
- Supports `--backend elevenlabs|gtts|chatterbox` (default: `elevenlabs`): `gtts` routes all dialogue voice stems through Google Translate TTS at no cost — flat single voice, useful for duration checks; `chatterbox` uses local Chatterbox TTS with per-character zero-shot voice cloning from `voice_refs/<key>.wav` reference clips — near-production quality, GPU-accelerated, free after setup; SFX/music/ambience generation is unaffected by `--backend`; eleven_v3 inline tags are stripped before gTTS/Chatterbox calls
- `--backend chatterbox` options: `--chatterbox-python PATH` (default: auto-detect `./venv-chatterbox/bin/python3`); `--voice-refs DIR` (default: `voice_refs/`) for per-speaker `.wav` reference clips; `--exaggeration FLOAT` emotion level 0.0–1.0 (default: 0.5); missing voice refs fall back to Chatterbox default voice
- Intro music (`INTRO MUSIC` source entry): trimmed at copy time using `play_duration` percentage from sfx config, so the stem file reflects the actual playback length
- Skips stems that already exist on disk

### Stage 3: Audio Assembly

`XILP003_audio_assembly.py` — Two-pass multi-track mix into a final master MP3.

```bash
xil assemble --episode S01E01
xil assemble --episode S01E01 --parsed parsed/parsed_<slug>_S01E01.json
```

- When a parsed script JSON is available (auto-derived or via `--parsed`), runs a two-pass multi-track mix:
  - **Foreground pass**: dialogue + one-shot SFX/BEAT stems concatenated sequentially
  - **Background pass**: AMBIENCE stems looped across scene boundaries (ducked -10 dB); MUSIC stings overlaid at cue points (-6 dB)
  - Foreground and background combined via `AudioSegment.overlay()`
- Falls back to single-pass sequential concatenation when no parsed JSON is found
- Stem classification uses `direction_type` from the parsed JSON, keyed by seq number in the filename
- Shared mixing logic lives in `mix_common.py` — also used by XILP005
- Applies per-speaker effects (pan, audio filters) from cast config; `filter` field accepts `false`/`null` (none), `true`/`"phone"` (phone filter), `"vintage"` (vintage filter), or a comma-separated combination such as `"vintage,phone"`
- Applies scene-scoped vintage filter to all dialogue in scenes listed in `sfx_config.vintage_scenes`; applied after the per-speaker filter chain
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- Supports `--output` to set the master MP3 path (default: `<slug>_S01E01_master.mp3`)
- `--gap-ms N` sets the silence gap between foreground stems in milliseconds (default: 600); reducing to 200–300 can shorten episode runtime by 1.5–2 minutes
- No ElevenLabs API key required — safe to re-run freely

### Stage 4: Studio Project Onboarding

`XILP004_studio_onboard.py` — Creates an ElevenLabs Studio project from parsed episode data.

```bash
xil studio-onboard --episode S01E02 --dry-run
xil studio-onboard --episode S01E02
xil studio-onboard --episode S01E02 --quality high
```

- `--episode` or `--tag` (one required) derives `parsed_<slug>_S01E02.json` and `cast_<slug>_S01E02.json`
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- Builds `from_content_json` payload for the Studio Projects API with per-node `voice_id` assignments
- Solves the speaker-name problem: voice assignments are embedded directly — no speaker names in TTS text
- Content mapping: sections → chapters, dialogue → `tts_node` blocks, scene headers → `h2` blocks, directions → skipped
- `--dry-run` displays chapter/block summary with voice assignments without calling the API
- `--quality` sets quality preset (standard/high/ultra/ultra_lossless, default: standard)
- `--model` sets TTS model (default: eleven_v3)
- Validates no TBD voice_ids in cast config before proceeding
- Requires `ELEVENLABS_API_KEY` env var for non-dry-run mode

### Stage 5: DAW Layer Export

`XILP005_daw_export.py` — Exports four isolated, full-length WAV layers for human mixing in Audacity.

```bash
xil daw --episode S01E01 --dry-run
xil daw --episode S01E01
xil daw --episode S01E01 --macro
xil daw --episode S01E01 --output-dir exports/S01E01/
xil daw --episode S01E01 --dry-run --timeline
xil daw --episode S01E01 --timeline --timeline-html
```

- `--episode` or `--tag` (one required) derives `cast_<slug>_S01E01.json` and `parsed/parsed_<slug>_S01E01.json`
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- Outputs four WAV files to `daw/{TAG}/` — all identical duration, all aligned at t=0:
  - `{TAG}_layer_dialogue.wav` — spoken dialogue (audio filter chain + pan applied per speaker)
  - `{TAG}_layer_ambience.wav` — environmental background looped to fill scene durations
  - `{TAG}_layer_music.wav` — music stings/themes at cue positions
  - `{TAG}_layer_sfx.wav` — one-shot SFX and BEAT silences
- Each WAV is tagged with ID3 metadata (Album, Genre, Year, Title, Artist) via `tag_wav()` from `sfx_common.py`
- Generates four Audacity label track files (`{TAG}_labels_dialogue.txt`, etc.) — tab-separated start/end/text
- Generates `{TAG}_open_in_audacity.py` — prints WAV import instructions (labels listed separately as optional)
- `--macro` writes an Audacity macro (`THE413_{TAG}.txt`) to `%APPDATA%\audacity\Macros\` for one-click WAV import via `Tools > Macros`
- `--dry-run` shows stem counts and output paths without writing files
- `--gap-ms N` sets the silence gap between foreground stems in milliseconds (default: 600); reducing to 200–300 can shorten episode runtime by 1.5–2 minutes
- `--save-aup3` includes a `SaveProject2` command in the generated `{TAG}_open_in_audacity.py` helper script (requires mod-script-pipe in Audacity)
- `--timeline` prints an ASCII multitrack timeline to stdout (works with `--dry-run` via fast mutagen header reads)
- `--timeline-html` writes a self-contained interactive HTML timeline to `daw/{TAG}/{TAG}_timeline.html` (hover tooltips, Ctrl+scroll zoom)
- Preamble/postamble stems are picked up automatically via `collect_stem_plans()` — they are regular seq-numbered entries in the parsed JSON (no special negative-seq handling)
- No ElevenLabs API key required — no API calls made
- Shared mixing logic imported from `mix_common.py`; visualization via `timeline_viz.py`

### Stage 6: Stem Migration (Punch-In Workflow)

`XILP007_stem_migrator.py` — Migrates episode stems when a parsed script is revised. Compares an old and new parsed JSON, copies unchanged stems to their new seq-numbered filenames, and reports which entries need fresh TTS/SFX generation. Run XILP002 afterwards to fill only the gaps.

```bash
xil migrate --episode S02E03 --dry-run
xil migrate --episode S02E03
xil migrate \
    --old parsed/orig_parsed_<slug>_S02E03.json \
    --new parsed/parsed_<slug>_S02E03.json \
    --stems stems/S02E03 [--dry-run] [--strict]
```

- `--episode TAG` derives `--old` (`parsed/orig_parsed_<slug>_{TAG}.json`), `--new` (`parsed/parsed_<slug>_{TAG}.json`), and `--stems` (`stems/{TAG}`) automatically
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- `--orig-prefix` (default: `orig_`) sets the filename prefix for the old parsed JSON
- `--dry-run` — shows the full plan without copying any files
- `--strict` — exact text match only; default is **fuzzy** (normalises em-dash, ellipsis, curly quotes so punctuation-only edits don't force unnecessary regen)
- `--quiet` — prints only the summary, not per-stem details
- Status codes printed per stem: `COPY` (unchanged, will be/was copied), `SPEAKER` (text matches but speaker reassigned → regen), `NEW` (no old entry matches → generate), `MISSING` (match found but old file absent → generate); each status line is followed by a truncated text snippet (first 55 chars) for visual content verification
- Two-phase matching: phase 1 matches on (text, speaker); phase 2 (dialogue only) falls back to text-only to detect speaker reassignments
- After running (without `--dry-run`), run `XILP002_producer.py --episode TAG` — it skips stems already on disk, so only SPEAKER/NEW/MISSING slots get API calls
- No ElevenLabs API key required — no API calls made

### Stage 7: Stale Stem Cleanup

`XILP008_stale_stem_cleanup.py` — Removes stale stems left behind after a parsed script revision and stem migration. After XILP007 copies unchanged stems to new seq-numbered filenames, old stems whose seq numbers now map to a different entry type remain on disk. This script finds and deletes them.

```bash
xil cleanup --episode S02E03 --dry-run
xil cleanup --episode S02E03
xil cleanup \
    --parsed parsed/parsed_<slug>_S02E03.json \
    --stems stems/S02E03 [--dry-run]
```

- `--episode TAG` derives `--parsed` (`parsed/parsed_<slug>_{TAG}.json`) and `--stems` (`stems/{TAG}`) automatically
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- `--parsed` and `--stems` override individual paths (both required if `--episode` is omitted)
- `--dry-run` — lists stale stems without deleting them
- A stem is stale when its filename disagrees with the current parsed entry: entry type is a header (`section_header`/`scene_header`), `_sfx` suffix but entry is now `dialogue`, speaker suffix but entry is now `direction`, dialogue stem whose speaker suffix doesn't match the parsed speaker, or seq not present in parsed JSON at all
- Duplicate detection: when multiple stems share the same seq, keeps only the one whose basename matches the expected `{seq}_{section}[-{scene}]_{speaker|sfx}` pattern
- Uses `extract_seq()` and `load_entries_index()` from `mix_common.py`
- No ElevenLabs API key required — no API calls made

### Stage 8: Studio Export Import

`XILP010_studio_import.py` — Extracts dialogue and direction stems from an ElevenLabs Studio export ZIP and renames them to the pipeline's stem naming convention.

```bash
xil import --episode S02E02 --zip "ElevenLabs_exports/export.zip" --dry-run
xil import --episode S02E02 --zip "ElevenLabs_exports/export.zip"
xil import --episode S02E02 --zip "ElevenLabs_exports/export.zip" --gen-sfx --gen-music --gen-beats
xil import --episode S02E02 --zip "ElevenLabs_exports/export.zip" --all --force
```

- `--episode TAG` (required) derives parsed JSON path and stems output directory
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- `--zip PATH` (required) path to the ElevenLabs Studio export ZIP
- `--parsed PATH` overrides parsed JSON path (default: `parsed/parsed_<slug>_{TAG}.json`)
- `--stems-dir PATH` overrides stems output directory (default: `stems/{TAG}`)
- `--dry-run` — shows extraction plan without writing files
- `--force` — overwrites existing stems on disk (default: skip if exists)
- `--gen-sfx` — include SFX direction entries (extracted as `_sfx` stems)
- `--gen-music` — include MUSIC direction entries (extracted as `_sfx` stems)
- `--gen-beats` — include BEAT direction entries (extracted as `_sfx` stems)
- `--all` — include all direction types (SFX, MUSIC, BEAT, AMBIENCE); headers are always skipped
- Dialogue entries are always extracted; direction entries require one of the `--gen-*` or `--all` flags
- ElevenLabs Studio exports one MP3 per parsed entry (`NNN_Chapter N.mp3`)
- Reuses `make_stem_name()` from XILP007 for canonical stem filename generation
- No ElevenLabs API key required — no API calls made

### Stage 9: Final Master MP3 Export

`XILP011_master_export.py` — Overlays the four DAW layer WAVs from XILP005 into a single podcast-ready MP3.

```bash
xil master --episode S02E03 --dry-run
xil master --episode S02E03
xil master --episode S02E03 --show "Night Owls"
```

- `--episode` or `--tag` (one required) derives DAW layer paths and cast config
- `--show` overrides the show name (default: from `project.json`)
- `--daw-dir` overrides the DAW layer directory (default: `daw/<TAG>/`)
- `--output` overrides the output MP3 path (default: `masters/<TAG>_<slug>_<YYYY-MM-DD>.mp3`)
- `--dry-run` shows layer summary without writing files
- Output format: stereo, 48 kHz, VBR MP3 (~145–185 kbps, LAME quality 2)
- Output filename: `S02E03_the413_2026-03-24.mp3` (episode tag, show slug, run date)
- Overlays all four layers at unity gain (XILP005 handles mix balance)
- Reads cast config for ID3 metadata (album, title, artist)
- No ElevenLabs API key required — no API calls made

### Stage 10: Social Media Post Draft Generator

`XILP012_publish.py` — Reads a parsed episode JSON, builds a structured episode summary, and calls the Claude API (Haiku) to produce three ready-to-edit Facebook/Instagram post variants. Output is an editable markdown file.

```bash
xil publish --episode S04E01 --dry-run
xil publish --episode S04E01
xil publish --episode S04E01 --platform instagram
xil publish --all
```

- `--episode` or `--tag` (one required unless `--all` is used) derives parsed JSON and cast config
- `--show` overrides the show name (default: from `project.json`)
- `--platform facebook|instagram` — affects post length/style guidance (default: `facebook`)
- `--dry-run` — prints the Claude prompt and estimated token count; no API call, no file written
- `--all` — generate posts for every parsed episode under the current show slug (retroactive batch)
- `--model` — override the Claude model ID (default: `claude-haiku-4-5-20251001`)
- Output: `posts/{slug}/{tag}_posts.md` — editable markdown with three variants
- Three post variants: **Hype** (new episode teaser, no spoilers past cold open), **Quote** (pull quote from cold open + tune-in CTA), **Spotlight** (one cast member feature, cycles by episode number mod cast count)
- Requires `ANTHROPIC_API_KEY` environment variable (only for non-dry-run mode)
- Requires `[publish]` optional extra: `pip install 'xil-pipeline[publish]'` (installs `anthropic>=0.40`)
- System prompt is tagged with `cache_control: ephemeral` to minimize cost on `--all` batch runs
- No ElevenLabs API key required — no ElevenLabs calls made

## ElevenLabs API Cost Controls

Every script that calls the API includes three guard functions (duplicated per file, not shared):

- `check_elevenlabs_quota()` — displays current character usage vs limit
- `has_enough_characters(text)` — per-line quota check before each API call
- `get_best_model_for_budget()` — always returns `eleven_v3`; logs a warning when balance is low (no longer falls back to `eleven_flash_v2_5`, which does not support `[pause]` and other native audio tags)

Always use `--dry-run` before running voice generation on a new script to verify TTS character budget.

## File Naming Convention

All scripts live under `src/xil_pipeline/` and are installed as `xil-*` console entry points plus a unified `xil` command via `pyproject.toml` (example: `xil parse ...` routes to `xil-parse`). Scripts use prefix `XIL` (ElevenLabs, avoiding numeric prefixes). The suffix pattern is:

- `XILP000_*` — pre-flight script scanner (no API, no side effects)
- `XILU001_*` — voice discovery (browse ElevenLabs voices; `--update-cast` back-fills role/language_code into a cast JSON)
- `XILU002_*` — standalone SFX stem generation
- `XILU003_*` — CSV + SFX/cast annotation utility (joins parsed episode CSV with SFX JSON and cast JSON for review)
- `XILU004_*` — voice sample generator (audition cast voices)
- `XILU005_*` — SFX library discovery (`--local` scans `SFX/` directory, default; `--api` queries ElevenLabs history)
- `XILU006_*` — parsed JSON splice utility (insert/delete entries with automatic seq renumbering)
- `XILU007_*` — MP3 hash utility (compute SHA256 checksums for stem files)
- `XILU008_*` — stem log report (parse daily logs → chronological stem generation CSV with backend/model/hash)
- `XILU009_*` — workspace migration tool (move pre-0.1.8 legacy layout to normalized `configs/`/`parsed/`/`daw/` structure)
- `xil_gui.py` — Gradio web dashboard (`xil-gui` entry point); requires `[gui]` extra; five tabs: Episodes, Audio Preview, Run Stage, Setup (with content type selector), Timeline
- `XILP001_*` — script parser
- `XILP002_*` — voice generation (ElevenLabs TTS)
- `XILP003_*` — audio assembly (stems → master MP3, two-pass multi-track mix)
- `XILP004_*` — Studio project onboarding (ElevenLabs Studio Projects API)
- `XILP005_*` — DAW layer export (stems → per-layer WAVs for Audacity)
- `XILP006_*` — cues sheet ingester (cues markdown → SFX library + sfx config enrichment)
- `XILP007_*` — stem migrator (diff old vs new parsed JSON, copy unchanged stems, report what needs regen); `--dry-run` report shows truncated text snippets alongside COPY/NEW/SPEAKER/MISSING entries for visual content verification without cross-referencing JSON files
- `XILP008_*` — stale stem cleanup (delete stems whose seq no longer matches the current parsed JSON)
- `XILP009_*` — reverse script generator (parsed JSON → production script markdown)
- `XILP010_*` — Studio export importer (ElevenLabs Studio ZIP → pipeline stems)
- `XILP011_*` — final master MP3 export (overlay 4 DAW layer WAVs → single stereo 48 kHz VBR MP3)
- `XILP012_*` — social media post draft generator (parsed JSON + Claude Haiku → `posts/{slug}/{tag}_posts.md`; 3 variants: Hype, Quote, Spotlight; requires `[publish]` extra + `ANTHROPIC_API_KEY`)
- `mix_common.py` — shared mixing utilities (timeline, layer builders, fast label helpers) used by XILP003 and XILP005; `StemPlan.scene` (str|None): scene label from parsed JSON, used for scene-scoped vintage filter; `StemPlan.loop` field: `True` (default) tiles audio, `False` plays once up to scene boundary; `StemPlan.pre_trimmed` flag: skips play_duration trim for source-based stems already trimmed at copy time; `StemPlan.volume_percentage` (float|None): volume as a percentage (100 = unity, None = no change); `StemPlan.ramp_in_seconds` / `StemPlan.ramp_out_seconds`: fade durations in seconds (None = no fade); `_resolve_audio_params()` resolves volume/ramp from per-effect config or category defaults for MUSIC, AMBIENCE, SFX, and BEAT direction types; `volume_percentage`, `ramp_in_seconds`, and `ramp_out_seconds` each fall back to the global key when no category-specific key exists (e.g. SFX/MUSIC when `sfx_volume_percentage`/`music_ramp_in_seconds` are absent from the config defaults); `collect_stem_plans()` skips stale stems (header entries, type mismatch, speaker mismatch), deduplicates by seq number, and injects synthetic stop-marker `StemPlan` entries (filepath="") for `AMBIENCE: STOP` and `AMBIENCE: * FADES OUT` directives found in the entries index; `build_sfx_layer()` and `build_foreground()` apply `volume_percentage` to SFX/BEAT stems; `build_ambience_layer()` skips corrupt or unreadable stem files with a warning rather than crashing; `apply_vintage_filter()` applies a 1960s-era HF roll-off + 1 dB reduction; `_apply_speaker_filters(segment, filter_val)` resolves the cast config `filter` string and applies the named filter chain (`false`/`None` = none, `true`/`"phone"` = phone, `"vintage"` = vintage, `"vintage,phone"` = both); `_vf_engaged_seqs(stem_plans)` returns the set of dialogue seq numbers that fall within a `VINTAGE FILTER: ENGAGES`…`VINTAGE FILTER: DISENGAGES` span — used by `build_foreground()` and `build_dialogue_layer()` to apply the vintage EQ to dialogue within those spans (script-direction spans take precedence over `vintage_scenes` list fallback)
- `sfx_common.py` — shared SFX library management, ID3 tagging (`tag_mp3`, `tag_wav`), effect generation; `ensure_shared_asset()` retries on 429 rate-limit errors (up to 5 times, linear backoff); `load_sfx_entries()` accepts `direction_types` filter set, returns `direction_type` field in each entry dict, skips entries with `duration_seconds=0.0`; `dry_run_sfx()` shows per-category credit subtotals in the SUMMARY block
- `timeline_viz.py` — multitrack timeline visualization; `render_terminal_timeline()` (ASCII) and `render_html_timeline()` (interactive HTML); no pydub dependency; HTML bar badges: `ri` (↑ ramp in, left), `ro` (↓ ramp out, right-top), `pd` (% play duration, center), `vb` (🔊 volume%, right-bottom, shown when `volume_pct != 100`); applies to music, ambience, and SFX spans
- `models.py` — Pydantic data models plus `get_workspace_root()` (respects `XIL_PROJECTROOT` env var), `show_slug()`, `derive_paths()`, `resolve_slug()` for dynamic show-based path derivation; `DEFAULT_SLUG = "sample"` fallback; `ProjectConfig` model with `type`/`tag_format` fields; `TYPE_DEFAULTS` dict with gap_ms and stability per content type; `derive_paths_legacy()` returns pre-0.1.8 paths anchored to workspace root (used by migration tool); `derive_paths()` auto-detects layout (legacy if root cast config exists, normalized otherwise); `load_project_config()` / `resolve_project_type()` helpers
- `xil.py` — unified dispatcher that maps subcommands (`scan`, `parse`, `produce`, etc.) to existing module `main()` entry points; prints command list on `xil --help`; `xil-*` commands remain supported
- `xil_init.py` — project scaffolding; `--type podcast|audiobook|drama|special` selects sample script, speakers.json, and project.json `type` field; creates new normalized workspace layout (`configs/{slug}/`)

## Cast Configuration

`cast_<slug>_S01E01.json` (e.g. `cast_the413_S01E01.json`) contains show-level metadata (`show`, `season`, `episode`, `title`) and a `cast` dict mapping character keys to settings:

```json
{
  "show": "THE 413", "season": 1, "episode": 1, "title": "The Holiday Shift",
  "cast": {
    "adam": { "full_name": "Adam Santos", "voice_id": "...", "pan": 0.0, "filter": false, "role": "Host/Narrator" },
    "mr_patterson": { "full_name": "Mr. Patterson", "voice_id": "...", "pan": -0.3, "filter": "phone", "role": "Caller" },
    "young_adam": { "full_name": "Young Adam", "voice_id": "...", "pan": 0.0, "filter": "vintage", "role": "Flashback" }
  }
}
```

Voice IDs are discovered via `XILU001_discover_voices_T2S.py` (filters to premade category).

Optional `preamble` and `postamble` blocks define the speaker and speed for those sections. The actual dialogue text lives in the script (PREAMBLE/POSTAMBLE sections); the cast config only needs `speaker` and `speed`:

```json
{
  "preamble": {
    "speaker": "tina",
    "speed": 0.85
  },
  "postamble": {
    "speaker": "tina",
    "speed": 0.85
  }
}
```

Intro/outro music lives in the SFX config under `"INTRO MUSIC"` / `"OUTRO MUSIC"` keys — not in the cast config. All TTS generation uses `eleven_v3` unconditionally.

## SFX Configuration

`sfx_<slug>_S01E01.json` (e.g. `sfx_the413_S01E01.json`) maps parsed direction entry text to ElevenLabs Sound Effects API parameters:

```json
{
  "show": "THE 413", "season": 1, "episode": 1,
  "defaults": { "prompt_influence": 0.3 },
  "effects": {
    "INTRO MUSIC": { "source": "SFX/The Porch Light.mp3" },
    "SFX: PHONE BUZZING": { "prompt": "Phone vibrating buzz", "duration_seconds": 2.0 },
    "BEAT": { "type": "silence", "duration_seconds": 1.0 }
  }
}
```

- Keys match the `text` field of parsed direction entries exactly
- `"INTRO MUSIC"` / `"OUTRO MUSIC"` are reserved keys for preamble/postamble music; XILP002 reads their `source` field to copy the audio file into the appropriate seq-numbered stem — no API generation
- `type: "sfx"` (default) entries call `client.text_to_sound_effects.convert()` with the `prompt`
- `type: "silence"` entries (BEAT/LONG BEAT) generate local silent audio — no API call
- `loop: false` entries play the audio file once up to the scene boundary (no tiling); `loop: true` (default) tiles the file to fill the full scene duration
- `volume_percentage` — per-effect volume as a percentage (100 = unity, 50 = half volume); applies to SFX, BEAT, MUSIC, and AMBIENCE entries; overrides the category default (`sfx_volume_percentage`, `music_volume_percentage`, `ambience_volume_percentage`) in the `defaults` block
- `play_duration` — percentage of file to play (e.g. `45` = play 45% of file duration); for INTRO MUSIC, the trim is applied when copying to the stem file so all downstream tools see the correct duration
- Stop markers: `AMBIENCE: STOP` and `AMBIENCE: * FADES OUT` entries use `type: "silence", duration_seconds: 0.0`; they inject a boundary marker into the mixing timeline without generating audio
- `vintage_scenes` — optional top-level list of scene labels (e.g. `["scene-3", "scene-4", "scene-5"]`); all dialogue in those scenes receives the vintage audio filter (HF roll-off + 1 dB reduction) during assembly; applied after per-speaker filters; omit or leave empty for no vintage treatment; the same scene labels used in the parsed JSON `scene` field
- SFX stems use `_sfx` suffix: `002_cold-open_sfx.mp3`

### Shared SFX Library

Each unique sound effect is generated **once** into the `SFX/` directory as a shared asset (e.g. `SFX/beat.mp3`, `SFX/sfx_phone-buzzing.mp3`). Episode stems in `stems/<slug>/<TAG>/` are copies of these shared assets with sequence-numbered filenames. This avoids regenerating the same effect for repeated uses (e.g. BEAT appears 26 times in S01E01). See `docs/sfx-reuse-guide.md` for a workflow guide on maximizing SFX reuse and minimizing API credit spend.

- Shared asset naming: `slugify_effect_key()` in `sfx_common.py` converts direction text to filesystem-safe slugs
- `--dry-run` shows three statuses: `EXISTS` (episode stem on disk), `CACHED` (shared asset exists, will be copied), `NEW` (needs API generation)
- Common SFX functions live in `sfx_common.py` — both XILU002 and XILP002 delegate to it
- `tag_mp3()` writes ID3 metadata (Album, Genre, Year, Title, Artist, Lyrics) to MP3 stems
- `tag_wav()` writes ID3 metadata (Album, Genre, Year, Title, Artist) to WAV layer exports

### Standalone SFX Utility

`XILU002_generate_SFX.py` — Generates SFX stems independently of XILP002 voice generation.

```bash
xil sfx --episode S01E01 --dry-run
xil sfx --episode S01E01 --gen-sfx
xil sfx --episode S01E01 --gen-music
xil sfx --episode S01E01 --gen-ambience
xil sfx --episode S01E01 --max-duration 5.0
xil sfx --episode S01E01 --local-only
xil sfx --episode S01E01
```

- `--episode` or `--tag` (one required) derives `cast_<slug>_S01E01.json` and `sfx_<slug>_S01E01.json`
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- Reads: parsed script JSON + SFX config + cast config (for episode tag)
- Outputs: shared assets to `SFX/`, episode stems to `stems/<slug>/<TAG>/`
- `--dry-run` shows EXISTS/CACHED/NEW status per stem with credit estimates
- `--gen-sfx`, `--gen-music`, `--gen-ambience` filter generation to the specified categories; omitting all three processes all categories
- `--dry-run` SUMMARY now shows per-category credit subtotals (MUSIC / AMBIENCE / SFX / silence)
- `--max-duration N` filters to effects ≤ N seconds (controls API credit spend)
- `--local-only` skips any effect not already present in `SFX/`; only CACHED assets and silence entries are placed, no API calls made
- 429 rate-limit errors are retried automatically up to 5 times with linear backoff (10s, 20s, 30s, 40s, 50s)
- Skips stems that already exist on disk

### CSV Annotation Utility

`XILU003_csv_sfx_join.py` — Joins a parsed episode CSV with the SFX JSON and cast JSON, producing an annotated review CSV with SFX prompt, duration, and cast metadata columns appended alongside each dialogue and direction entry.

```bash
xil csv-join --episode S02E03
xil csv-join --episode S02E03 --output my_review.csv
```

- `--episode` or `--tag` (one required) derives `parsed/parsed_<slug>_{TAG}.csv`, `sfx_<slug>_{TAG}.json`, and `cast_<slug>_{TAG}.json`
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- `--csv`, `--sfx`, `--cast` override individual input paths
- `--output` overrides the output CSV path (default: `parsed/annotated_<slug>_{TAG}.csv`)
- No API key required — read-only join utility

### Voice Sample Utility

`XILU004_sample_voices_T2S.py` — Generates a short TTS sample for each cast member to audition voice assignments.

```bash
xil sample --episode S02E03 --dry-run
xil sample --episode S02E03
xil sample --episode S02E03 --backend chatterbox
xil sample --episode S02E03 --backend gtts
xil sample --episode S02E03 --force
```

- `--episode` or `--tag` (one required) or `--cast PATH` to specify the cast config
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- `--backend elevenlabs|gtts|chatterbox` (default: `elevenlabs`): selects TTS backend for sample generation
- Default sample text: `"I am {name} not yo momma"`; override with `--sample-text` (use `{name}` placeholder)
- Output: `voice_samples/{TAG}/{backend}/{actor}.mp3` — backend subdirectory enables side-by-side comparison
- Skips members with `voice_id=TBD` (ElevenLabs only); `--force` regenerates existing samples
- `--chatterbox-python PATH`, `--voice-refs DIR`, `--exaggeration FLOAT` — Chatterbox-specific options (same as `xil-produce`)
- Requires `ELEVENLABS_API_KEY` for `--backend elevenlabs`

### SFX Library Discovery

`XILU005_discover_SFX.py` — Lists and searches the local shared SFX asset library.

```bash
xil sfx-lib                    # local scan (default)
xil sfx-lib --local            # explicit local scan
xil sfx-lib --sfx-dir SFX/    # override local scan directory
xil sfx-lib --search "diner"   # filter by keyword
xil sfx-lib --json             # machine-readable output
xil sfx-lib --api              # attempt API history (not publicly accessible)
xil sfx-lib --api --all        # paginate full API history (default: most recent 100)
```

- Default mode: scans `SFX/` directory (equivalent to `--local`) and reports all assets with duration and file size
- `--local` / `--api` are mutually exclusive mode flags; `--local` is the default
- `--sfx-dir DIR` overrides the local scan directory (default: `SFX/`)
- `--search TEXT` filters results by case-insensitive substring match on filename/prompt
- `--json` outputs results as a JSON array
- `--verbose` / `-v` prints all metadata fields per asset
- `--api` attempts to query ElevenLabs sound generation history (endpoint is not publicly accessible as of March 2026 regardless of API key permissions)
- `--all` (API mode only) paginates through the full account history; default retrieves only the most recent 100 results
- `--export-kit [DIR]` generates an SFX inventory JSON (`sfx_inventory.json`) and copies the scriptwriter reference doc (`claude-scriptwriter-reference.md`) into DIR (default: current directory); attach both files to a Claude project as knowledge files to enable SFX-aware script writing

### Parsed JSON Splice Utility

`XILU006_splice_parsed.py` — Inserts entries into or deletes entries from a parsed episode JSON with automatic seq renumbering.

```bash
xil splice --episode S02E03 --insert-after 322 \
    --from-parsed parsed/parsed_the413_S02E02.json --from-seq-range 232-233 \
    --section post-interview --dry-run
xil splice --episode S02E03 --delete-seq-range 100-105 --dry-run
xil splice --episode S02E03 --insert-after 322 \
    --from-json new_entries.json
```

- `--episode` or `--tag` (one required) derives target parsed JSON path
- `--show` overrides the show name used for slug derivation (see Project Configuration)
- `--parsed PATH` overrides target parsed JSON path
- `--insert-after N` — seq number to insert after
- `--from-parsed PATH` + `--from-seq-range N-M` — extract entries from another parsed JSON by seq range
- `--from-json PATH` — read entries from a standalone JSON array file
- `--section` / `--scene` — override section/scene on inserted entries (default: inherit from insertion point)
- `--delete-seq-range N-M` — remove entries in range and renumber (can combine with insertion: delete first, then insert)
- `--dry-run` — show plan without writing files
- `--no-backup` — skip writing backup file
- `--quiet` — summary only, no per-entry detail
- Before modifying, writes `parsed/pre_splice_parsed_<slug>_<TAG>.json` as a backup (compatible with `XILP007 --orig-prefix pre_splice_`)
- Preamble entries (seq <= 0) are never renumbered or deleted
- Recomputes the `stats` block after modification
- No ElevenLabs API key required — no API calls made

### Stem Log Report

`XILU008_stem_log_report.py` — Parses daily pipeline log files into a chronological stem generation CSV. Useful for auditing what was generated, when, with which backend, and confirming SHA256 checksums.

```bash
xil-stem-log --episode S03E03
xil-stem-log --episode S03E03 --since 2026-04-01
xil-stem-log --slug the413
xil-stem-log --logs-dir logs/ --output stem_log_report.csv
xil-stem-log --episode S03E03 --audit
xil-stem-log --episode S03E03 --audit --audit-threshold 20
```

- `--episode TAG` (optional) filters records to a specific episode tag (e.g. `S03E03`); matched against `stem_path`
- `--slug SLUG` (optional) filters records to a specific show slug (e.g. `the413`); matched against `stem_path`
- `--logs-dir DIR` path to log directory (default: `logs/`)
- `--output PATH` output CSV path (default: `stem_log_report.csv`); use `-` for stdout
- `--since DATE` filter to logs on or after the given date (YYYY-MM-DD)
- `--show` print CSV to stdout (equivalent to `--output -`)
- `--audit` cross-references logged `char_count` values against the current parsed JSON and flags stems whose logged character count differs from the current text length by more than `--audit-threshold` percent (default: 10); useful for detecting stale stems after script edits
- `--audit-threshold N` percentage threshold for the `--audit` flag (default: 10)
- Parses `logs/xil_YYYY-MM-DD.log` files; three regex patterns match ElevenLabs, gTTS, and Chatterbox generation lines
- State machine: generation line → saved line → SHA256 line → emits one record
- `run_index` counter increments per `Phase 1: Generating` marker, grouping stems by production run
- Output columns: `log_date`, `log_file`, `run_index`, `log_line`, `seq`, `speaker`, `backend`, `char_count`, `stem_path`, `stem_filename`, `sha256`
- No ElevenLabs API key required — reads local log files only

### Web Dashboard

`xil_gui.py` — Gradio browser dashboard that supplements the CLI. Five tabs: **Episodes** (workspace overview), **Audio Preview** (browse and play stems), **Run Stage** (launch pipeline stages with live log streaming), **Setup** (initialize a new workspace with content-type selector), **Timeline** (interactive HTML timeline). Requires the `[gui]` optional extra.

```bash
pip install 'xil-pipeline[gui]'
xil-gui                        # opens http://localhost:7860
xil-gui --share                # generates a public ngrok URL for partner access (72h, no auth)
xil-gui --port 8080            # custom port
xil gui                        # via unified dispatcher
```

Episode detection checks both legacy root `cast_{slug}_{tag}.json` and normalized `configs/{slug}/cast_{tag}.json` locations, so both old and new workspaces show up.

### Workspace Migration

`XILU009_migrate_workspace.py` — moves pre-0.1.8 workspace files to the normalized layout in one idempotent pass.

```bash
xil migrate-workspace --dry-run    # preview what would move
xil migrate-workspace              # execute moves
```

- **Episodes tab**: workspace overview — all detected episodes with parse/stems/DAW/master status
- **Audio Preview tab**: episode + filter selector (all/dialogue/sfx/music/ambience), stem dropdown, in-browser MP3 playback via `gr.Audio`; stem labels enriched from parsed JSON when available
- **Run Stage tab**: select episode + stage (assemble/daw/master/produce/parse), dry-run checkbox (default on), extra flags field; live stdout streaming via generator + `demo.queue()`
- **Timeline tab**: embeds the `daw/{TAG}/{TAG}_timeline.html` iframe if generated; prompts `xil daw --timeline-html` when absent
- `--share` uses Gradio's ngrok tunnel — open access, suitable for trusted collaborators during a session only
- `allowed_paths=[os.getcwd()]` enables Gradio to serve local MP3/WAV/HTML files
- Subprocess isolation: each stage run is a fresh `subprocess.Popen` so the GUI stays alive if a stage errors
- No ElevenLabs API key required for the GUI itself (stages may require it depending on flags)

## Developer/Maintainer Rules

Automated testing via Python and Bash serves as the fundamental mechanism for the Verification Loop. The project mandates that Claude must mention how it will verify its work before it begins any task.

Use tests for everything it implements:

- Determine which tests are appropriate; the model will then generate a test for every single feature it builds
- Test-Driven Development (TDD): A key best practice is implementing a verification-led technique where tests for a new feature are written first, followed by the actual code implementation

### Documentation Currency Rule

After executing any plan that changes pipeline behaviour, CLI flags, file formats, or module interfaces, **both** `CLAUDE.md` (root) and `docs/pipeline.md` must be updated to reflect those changes **before committing**. This applies equally to Claude and human contributors. Specifically:

- New CLI flags or flag removals → update the relevant stage description in CLAUDE.md and the corresponding section/sequence diagram in pipeline.md
- New module fields or dataclass additions → update `mix_common.py` / `sfx_common.py` bullet points in CLAUDE.md
- New SFX config keys or behaviours → update the SFX Configuration section
- New pipeline stages or utilities → add a XILP/XILU entry under File Naming Convention and a stage section in pipeline.md
- Any behavioural change visible to operators → update the relevant stage bullets in CLAUDE.md

If a plan is large enough to have its own plan file, tick this as the final step before closing the plan.

### Script Entry Point Style

Always use the `if __name__ == "__main__":` idiom. All application logic that would otherwise follow it must live inside a `main()` function — the dunder-main block must contain only the call to `main()`:

```python
def main():
    parser = argparse.ArgumentParser(...)
    args = parser.parse_args()
    # all application logic here

if __name__ == "__main__":
    main()
```

This keeps the `__main__` block to a single line, makes the entry point testable by calling `main()` directly, and prevents module-level side effects when the file is imported.

## Running Tests

```bash
pip install -e ".[all,dev]"
pytest tests/ -v
```

## Man Pages

Unix man pages for all 21 CLI commands are pre-generated and committed to `man/man1/`. They are installed automatically when the package is built into a wheel and installed via pip.

**Regenerating after CLI changes** (run whenever flags or descriptions change):

```bash
pip install -e ".[dev]"      # includes argparse-manpage
python docs/build_man.py  # regenerate all 20 argparse-based pages
# xil.1 is hand-crafted — edit man/man1/xil.1 directly when the dispatcher changes
```

Regenerate a single page: `python docs/build_man.py xil-parse`

Always commit the regenerated `.1` files alongside any CLI flag change. The `get_parser()` function in each module (extracted from `main()`) is what `build_man.py` calls to obtain the parser — keep it in sync with any `add_argument` changes.

**Post-install access on Debian** (for `pip install --user`):

```bash
# Pages land in ~/.local/share/man/man1/ — add to ~/.bashrc:
export MANPATH="$HOME/.local/share/man:$(manpath 2>/dev/null)"

# Then:
man xil-parse

# For apropos/whatis support:
mandb --user-db ~/.local/share/man
```

System-wide installs (`sudo pip install`) land in `/usr/local/share/man/man1/` and are indexed by default (`sudo mandb` to refresh).

## Key Directories

- `src/xil_pipeline/` — Python package (all pipeline and utility scripts, shared modules)
- `tests/` — Automated test suite (pytest)
- `scripts/` — Source markdown production scripts (authored manually)
- `parsed/` — Parser JSON output (generated, cacheable)
- `cues/` — Sound cues & music prompts markdown files (authored manually); `cues_manifest_<TAG>.json` generated by XILP006
- `stems/<slug>/<TAG>/` — Individual voice/SFX audio files per episode (generated, expensive to recreate)
- `SFX/` — Shared SFX asset library (generated once, reused across episodes); cues-sheet assets named by asset ID (e.g. `sfx-boots-stamp-01.mp3`)
- `daw/<TAG>/` — Per-layer WAV exports for DAW mixing (generated by XILP005)
- `venv/` — Python virtualenv (do not commit)
