Metadata-Version: 2.5
Name: vexy-speakup
Version: 1.0.0
Summary: Diarize, restore, level and master recorded speech with local VST3 plugins, keeping the video timeline
Project-URL: Repository, https://github.com/vexyart/vexy-speakup
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Environment :: Console
Classifier: Operating System :: MacOS
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Requires-Python: >=3.12
Requires-Dist: fire<0.8,>=0.7.1
Requires-Dist: numpy<3,>=2
Requires-Dist: pedalboard==0.9.26
Requires-Dist: pyloudnorm<0.3,>=0.2
Requires-Dist: soundfile<0.14,>=0.13.1
Provides-Extra: test
Requires-Dist: build>=1.2; extra == 'test'
Requires-Dist: pytest-cov>=6; extra == 'test'
Requires-Dist: pytest>=8; extra == 'test'
Requires-Dist: ruff>=0.12; extra == 'test'
Requires-Dist: twine>=6; extra == 'test'
Description-Content-Type: text/markdown

<!-- this_file: README.md -->
# vexy-speakup

Restore a meeting recording with your installed VST3 speech plugins, one speaker
at a time, and get back the same file with the original video timeline intact.

```bash
uv tool install vexy-speakup
# From a checkout before the first release:
uv tool install .
vexy-speakup run meeting.mp4 --profile_name=dx65 --verbose
```

It requires macOS, Python 3.12+ and FFmpeg/ffprobe on PATH. Rendering uses the
tested Pedalboard 0.9.26 wheel. The built-in profiles expect Accentize dxRevive
Pro, iZotope RX 12 Voice De-noise and Ozone 12 Maximizer. Plugins are read from
`/Library/Audio/Plug-Ins/VST3`. Set `VEXY_SPEAKUP_PLUGIN_ROOT` to use another
folder. Plugins and their licences are yours; none are distributed here.

## Commands

```bash
# Full audio or video, default work and output paths
vexy-speakup run meeting.m4a --profile_name=dx65

# Explicit paths and loudness
vexy-speakup run meeting.mp4 \
  --work_dir=work --output_path=result.mp4 \
  --target_lufs=-16 --true_peak=-2 --verbose

# Compare three processing attempts on one excerpt and retain model scores
vexy-speakup attempts meeting.m4a --work_dir=comparison --start=120 --duration=45

# Audio-only preview (video previews need a pretrimmed video)
vexy-speakup run meeting.m4a --start=120 --duration=45

# Export audio only from a video
vexy-speakup run meeting.mp4 --output_path=speech.wav

# Optional local DeepFilterNet3 CoreML before VST processing
vexy-speakup run meeting.m4a --ai=deepfilter

# Reuse or correct speaker annotations instead of rerunning a model
vexy-speakup run meeting.m4a --diarization_json=turns.json

# Alternate local diarizer; four speaker slots
vexy-speakup run meeting.m4a --diarizer=sortformer

# Inspect actual plugin controls before writing a custom chain
vexy-speakup inspect /Library/Audio/Plug-Ins/VST3/Accentize-dxRevivePro.vst3
vexy-speakup probe /Library/Audio/Plug-Ins/VST3/Accentize-dxRevivePro.vst3 probe.json
vexy-speakup inventory --output_path=inventory.json
vexy-speakup version
```

For `meeting.mp4`, the defaults are a work folder `meeting/` and an output file
`meeting_improved.mp4`, beside the input. Audio follows the same naming rule and
keeps its extension. Audio previews add `_preview`. An existing output is
refused unless `--force` is supplied. Every run gets a new directory inside the
work folder, so previous attempts and originals stay available.

## Local models

Diarization, DeepFilterNet and quality scoring call existing local adapters and
native hosts in isolated uv environments. Set `VEXY_SPEAKUP_TTS_ROOT` to the
folder holding `myasr/`, `myaud/` and `.external/`. Without it, you can still
process a recording with `--diarization_json=turns.json --evaluate=False`.

The default diarizer is the cached `mlx-community/Nemotron-3-Diarization-8bit`
conversion via `myasr/tools/diarize-mlx.py`. Sortformer uses the pinned Q8 GGUF
from the Hugging Face cache and the native `nemo-speech` host. No paid service is
used, and model access runs with `HF_HUB_OFFLINE=1`.

Evaluation invokes the cached `aud-metrics` environment and its DNSMOS/UTMOS
ONNX predictors. Override its Python with `--metrics_python=...` or disable
scoring with `--evaluate=False`. Missing predictors appear as errors in reports;
they never become synthetic scores. Normal runs score up to three matching
30-second speech windows.

## Profiles and custom chains

`dx35` is the conservative default: dxRevive at 35%, followed by Ozone Maximizer.
`dx65` raises restoration to 65%; it led DNSMOS on the tested Zoom excerpt.
`rx6-dx35` adds RX Voice De-noise at 6 dB before dxRevive.
`rx6` uses RX alone. `bypass` skips VSTs but keeps the level passes.

Use `--config_path=custom-chain.json` to replace the chains. The bundled
[example](examples/custom-chain.example.json) gives speaker `0` a lighter
dxRevive setting and mouth de-clicking. `speech` is the shared default chain,
`speakers` overrides individual speaker IDs, and `master` runs continuously after
assembly. Each plugin entry accepts `path`, `parameters`, optional `plugin_name`
for a multi-plugin bundle, and optional `preset` for a saved `.vstpreset` file.
Parameter names are validated against the installed plugin; unknown names fail.
Saved presets should come from your own host and installed plugin version.

## Processing and files

1. FFmpeg extracts stereo float PCM at 48 kHz. A delayed video audio start
   becomes leading silence. Diarization receives the original signal; optional
   AI restoration happens afterwards.
2. The diarizer processes the complete audio with its own bounded context, so
   speaker IDs are not restarted for each fragment.
3. The timeline is partitioned into disjoint speech, overlap and background
   intervals of at most 30 seconds (`--max_chunk`). Each speech fragment
   carries half a second of context at each edge.
4. One isolated plugin worker (`python -m vexy_speakup _render`) processes each
   speaker and resets the chain between fragments. Stereo is retained. Two
   seconds of padding drain plugin buffering; Pedalboard compensates reported
   plugin latency.
5. A duration-weighted median calibrates each speaker. Turn adjustments stay
   within 3 dB of that baseline, and total gain within ±12 dB. Silence is never
   amplified. Processing transitions blend with the source over 10 ms.
6. Timeline assembly writes every sample exactly once. Background gaps remain.
   Overlap is processed once as a mixed fragment under `speakers/_overlap`.
7. Two-pass R128 levelling precedes continuous mastering; a measured pass then
   sets final loudness and true peak. Export checks the encoded true peak and
   attenuates and re-encodes if lossy encoding overshoots.
8. For video, FFmpeg copies the original video packets and replaces the audio.

`speakers/<ID>/*.wav` are levelled timeline fragments; `*-context.wav` and
`*-render.wav` keep their processing context. `*-mastered.wav` are 24-bit cuts
from the final master, with final levels and the full-recording limiter
behaviour. `fragments.json` records original times, sample offsets, active
speakers and gains. Intermediate WAVs are float PCM and may exceed 0 dBFS;
`master.wav` and the published output are the peak-controlled deliverables.
A long recording can consume tens of GB of work files.

`turns.json`, diarization receipts, `speaker-gains.json`, VST parameter
receipts, `report.json`, `evaluation.json`, subprocess logs and `latest.json`
make each run reviewable. A failed run records its error and keeps its work
files. A successful output is published through a temporary file and rename.

## Limits

Predictor scores guide comparisons. They do not establish speaker accuracy,
transcript preservation or perceptual quality across a whole meeting. Diarized
fragments are not isolated voices during simultaneous speech; that would need
separate microphone tracks or a tested separation model. Speaker IDs are
anonymous until you label them.

Recordings, turn files and run folders are your private data. They are excluded
from the source and wheel allowlists, and nothing is uploaded.

## Development

```bash
./test.sh                 # pytest with coverage, then ruff
./build.sh                # tests, uv build, twine check; version from git tag
./publish.sh              # tag, build, uv publish to PyPI
```

Tag releases with `uvx gitnextver`. Untagged commits build as development
versions with a local segment, which PyPI rejects. The generated
`src/vexy_speakup/__version__.py` is ignored by git.

`./publish.sh` runs the tests, then `uvx gitnextver`, which commits all
changes, creates the next `vX.Y.Z` tag (`v1.0.0` when none exists) and pushes.
It then builds and runs `uv publish`. A rerun on a clean, already tagged HEAD
skips gitnextver.
