Metadata-Version: 2.4
Name: sprezzature-audio
Version: 1.1.0
Summary: Speech-to-text, speaker diarization, speaker identification, and multilingual caption translation. Local-first: Whisper, NeMo Sortformer, TitaNet, local LLM.
Author-email: Warith HARCHAOUI <warith.harchaoui@gmail.com>
License-Expression: BSD-3-Clause
Project-URL: Homepage, https://harchaoui.org/warith/sprezzature/
Project-URL: Repository, https://github.com/warith-harchaoui/sprezzature-audio
Keywords: whisper,diarization,speaker-identification,captions,subtitles,nemo,sprezzature,asr
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1
Requires-Dist: langdetect>=1.0.9
Provides-Extra: captions
Requires-Dist: vocal-helper>=0.6.0; extra == "captions"
Requires-Dist: audio-helper>=1.6.0; extra == "captions"
Requires-Dist: video-helper>=1.7.0; extra == "captions"
Provides-Extra: diarize
Requires-Dist: nemo_toolkit[asr]>=1.23.0; extra == "diarize"
Requires-Dist: numpy>=1.24; extra == "diarize"
Requires-Dist: audio-helper>=1.6.0; extra == "diarize"
Requires-Dist: video-helper>=1.7.0; extra == "diarize"
Provides-Extra: translate
Requires-Dist: best-engine-ai-helper>=0.4.0; extra == "translate"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Provides-Extra: all
Requires-Dist: sprezzature-audio[captions,diarize,translate]; extra == "all"
Provides-Extra: api
Requires-Dist: fastapi>=0.115; extra == "api"
Requires-Dist: uvicorn[standard]>=0.30; extra == "api"
Provides-Extra: mcp
Requires-Dist: sprezzature-audio[api]; extra == "mcp"
Requires-Dist: fastapi-mcp>=0.3; extra == "mcp"
Requires-Dist: mcp<2.0; extra == "mcp"
Dynamic: license-file

# sprezzature-audio

Local-first speech processing for the [sprezzature](https://harchaoui.org/warith/sprezzature/) stack.

Point it at a recording (a meeting, an interview, a lecture) and it hands back a transcript that says not just *what* was said, but *who* said it and *in which language*. Everything runs on your own machine: no audio leaves it, and no API key is needed. If you instead need to cut, resample, or clean up the audio signal itself (trim silence, separate a voice from background music), that is a different job, handled by a sibling package, [audio-helper](https://github.com/warith-harchaoui/audio-helper); this package starts once the audio is already usable and asks what was said in it.

## What it does

Six scripts, each one stage of the pipeline. A few terms recur throughout the table below, so here they are once, up front, rather than repeated at every mention:

- **ASR** (automatic speech recognition) is the technical name for speech-to-text: turning a sound wave into written words.
- **WebVTT** and **SRT** are two competing plain-text file formats for storing subtitles: a list of `[start time, end time, text]` triples. WebVTT is the web-standard one (what a `<video>` tag expects); SRT is older and more universally supported by video players.
- **RTTM** is a plain-text format from the speech-research world for recording *who spoke when*: one line per speaker turn, with a start time, a duration, and a speaker label.
- **NeMo** is NVIDIA's open-source toolkit for speech models; **Sortformer** and **TitaNet** are two specific NeMo models used here (diarization and speaker fingerprinting, explained below).

| Script | What it produces |
|---|---|
| `captions_from_whisper.py` | WebVTT, SRT, or a plain transcript, via a local Whisper model (through `vocal-helper`) |
| `diarize_from_nemo.py` | An RTTM file plus a JSON turn list, via NeMo's Sortformer model (up to 4 speakers) |
| `identify_from_titanet.py` | A speaker's identity, matched against a reference voice sample, via NeMo's TitaNet model |
| `caption_diarize.py` | The combined pipeline: transcript and speaker turns merged in one pass |
| `name_from_transcript.py` | A guess at each speaker's real name, read off the diarized transcript (pattern matching, with an optional LLM assist) |
| `translate_captions.py` | A translated copy of a VTT/SRT file, via a local LLM |

## Install

```sh
# Base (no ML dependencies)
pip install sprezzature-audio

# Add captioning (Whisper via vocal-helper)
pip install "sprezzature-audio[captions]"

# Add diarization and speaker ID (NeMo; install torch first, since the right
# build depends on your hardware: CUDA, Apple-silicon MPS, or plain CPU)
pip install torch
pip install "sprezzature-audio[diarize]"

# Add LLM translation (best-engine-ai-helper + a local Ollama server)
pip install "sprezzature-audio[translate]"

# Everything
pip install "sprezzature-audio[all]"
```

## Quick start

```sh
# Transcribe a video to WebVTT
python scripts/captions_from_whisper.py talk.mp4

# Same, as a plain-text transcript
python scripts/captions_from_whisper.py podcast.mp3 --format text

# Diarize an audio file: who spoke when
python scripts/diarize_from_nemo.py interview.wav

# Full pipeline: caption_diarize.py merges caption + diarization files
# already produced by the two steps above (it takes no media file itself)
python scripts/captions_from_whisper.py meeting.mp4
python scripts/diarize_from_nemo.py meeting.mp4
python scripts/caption_diarize.py --captions meeting.vtt --diarization meeting.diarization.json

# Guess speaker names from the diarized transcript
python scripts/name_from_transcript.py meeting.speakers.vtt

# Translate captions to French
python scripts/translate_captions.py talk.vtt --lang fr
```

## How this differs from audio-helper

`audio-helper` works at the **signal level**: converting formats, slicing a waveform, resampling, separating a voice from background music with Demucs. It has no notion of words; a silence and a sentence look the same to it.

`sprezzature-audio` works at the **content level**: it reads speech, attributes it to a speaker, and translates it. The two packages are meant to be used together, not as alternatives; `captions_from_whisper.py` in fact calls `audio-helper` internally, to extract a 16 kHz mono WAV file (the format Whisper expects) before it ever runs the speech model.

## Models used

| Task | Model | Backend |
|---|---|---|
| ASR (speech to text) | `large-v3-turbo` by default, or any other GGML-format Whisper weights (the compact file format `vocal-helper`'s underlying engine, whisper.cpp, expects) | vocal-helper / pywhispercpp |
| Diarization (who spoke when) | `nvidia/diar_sortformer_4spk-v1` | NeMo |
| Speaker ID (matching a voice to a reference sample) | `nvidia/speakerverification_en_titanet_large` | NeMo |
| Translation | Configured through the `SPREZZATURE_LLM_*` environment variables | best-engine-ai-helper |

## Environment variables

| Variable | Purpose |
|---|---|
| `SPREZZATURE_WHISPER_MODEL` | Override the Whisper model path or alias |
| `SPREZZATURE_CACHE_DIR` | Cache directory for Whisper weights and transcripts |
| `SPREZZATURE_NO_CACHE` | Set to any value to disable the transcript cache |
| `NEMO_DIAR_MODEL` | Override the NeMo diarization checkpoint |
| `SPREZZATURE_LLM_*` | LLM backend configuration (see best-engine-ai-helper) |

## License

BSD 3-Clause. See [LICENSE](https://github.com/warith-harchaoui/sprezzature-audio/blob/main/LICENSE).

## Author

Warith HARCHAOUI: [harchaoui.org/warith](https://harchaoui.org/warith/)
