Metadata-Version: 2.4
Name: eluate
Version: 0.0.3
Summary: Remove background music from videos - for accessibility and personal use
Author: anaxoniclabs
License-Expression: MIT
Project-URL: Homepage, https://github.com/anaxoniclabs/ELUATE
Project-URL: Repository, https://github.com/anaxoniclabs/ELUATE
Project-URL: Issues, https://github.com/anaxoniclabs/ELUATE/issues
Project-URL: Documentation, https://github.com/anaxoniclabs/ELUATE/blob/main/docs/api.md
Project-URL: Changelog, https://github.com/anaxoniclabs/ELUATE/blob/main/CHANGELOG.md
Keywords: video,audio,music-removal,documentary,ai,bandit,accessibility,audio-processing,source-separation
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: End Users/Desktop
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: rich>=13.0.0
Requires-Dist: torch>=2.1.0
Requires-Dist: torchaudio>=2.1.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: scipy>=1.11.0
Requires-Dist: librosa>=0.10.0
Requires-Dist: soundfile>=0.12.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: einops>=0.7.0
Requires-Dist: rotary-embedding-torch>=0.5.0
Requires-Dist: transformers>=4.35.0
Requires-Dist: huggingface-hub>=0.23.0
Requires-Dist: omegaconf>=2.2.0
Requires-Dist: ml-collections>=1.0.0
Requires-Dist: tqdm>=4.60.0
Requires-Dist: beartype>=0.14.0
Requires-Dist: spafe>=0.3.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Provides-Extra: cpu
Requires-Dist: torch>=2.1.0; extra == "cpu"
Requires-Dist: torchaudio>=2.1.0; extra == "cpu"
Provides-Extra: train
Requires-Dist: wandb>=0.15.0; extra == "train"
Requires-Dist: auraloss>=0.4.0; extra == "train"
Requires-Dist: pytorch-lightning>=2.0.0; extra == "train"
Requires-Dist: accelerate>=0.25.0; extra == "train"
Requires-Dist: loralib>=0.1.0; extra == "train"
Dynamic: license-file

# ELUATE

ELUATE is a command-line tool that removes the background music from a video and keeps the dialogue and sound effects. It's for people who want to watch documentaries, lectures, and other long-form video without a score running underneath: for religious reasons, or because hearing loss or an auditory-processing difference makes the music fight the narration.

I'm Muslim, and I kept hitting the same wall. The documentary I wanted to watch almost always had a score under it, and my usual move was to just skip it. So I built the thing I wanted: hand it a video, get the same video back with the music gone. The video stream is copied through untouched, so the picture stays bit-for-bit identical and only the audio changes.

```bash
eluate documentary.mp4
# → ~/Documents/ELUATE/documentary_eluted.mp4
```

There's a small Python API behind the same engine, so you can wire ELUATE into a batch job or another tool:

```python
import eluate

eluate.elute("documentary.mp4")
```

## Install

```bash
pip install eluate
```

ELUATE needs FFmpeg on your `PATH`. Install it with `brew install ffmpeg` on macOS or `sudo apt-get install -y ffmpeg` on Debian/Ubuntu. It's already there on Colab.

The first time you run `eluate`, it downloads a ~450 MB model checkpoint from [Zenodo](https://zenodo.org/records/12701995) into `~/.eluate/models/`. That happens once. To download it up front (and to make the Python API usable, since the API never downloads on its own), run:

```bash
eluate setup
eluate info     # check device, FFmpeg, and model status
```

On a machine with no CUDA or MPS, `pip install 'eluate[cpu]'` pins CPU-only PyTorch wheels and saves about a gigabyte.

A ready-to-run Colab notebook is at [`notebooks/eluate_colab_template.ipynb`](notebooks/eluate_colab_template.ipynb). Set the runtime to GPU (`Runtime -> Change runtime type -> GPU`) and run the cells top to bottom.

## Usage

```bash
eluate                                 # interactive: prompts for a file path
eluate video.mp4                       # single file → ~/Documents/ELUATE/
eluate video.mp4 -o cleaned.mp4        # custom output path
eluate --folder ./videos               # every video in a folder
eluate --folder ./videos --batch-size 10   # folder, pausing every 10 files
eluate --batch files.txt               # paths listed one per line in a file
eluate --checkpoint eng video.mp4      # English-tuned model instead of multilingual
eluate video.mp4 --device cpu          # force CPU (skip MPS/CUDA)
eluate video.mp4 --force               # skip the duration and disk-space prechecks
eluate video.mp4 --audio-codec alac    # write lossless audio instead of AAC
```

The folder and interactive modes accept these containers: `mp4`, `mkv`, `avi`, `mov`, `webm`, `flv`, `wmv`, `m4v`, `mpeg`, `mpg`, `3gp`, `3g2`, `ts`, `mts`, `m2ts`. Passing a file directly hands it to FFmpeg regardless of extension, so anything FFmpeg can demux will work.

Output audio defaults to AAC at 256k (`--audio-codec`, `--audio-bitrate`). Inference picks the best device automatically (CUDA, then MPS, then CPU); `--device` overrides it.

### Languages

The model ships per-language checkpoints: `multi` (default), `eng`, `deu`, `fra`, `spa`, `cmn`, `fao`. Switch with `--checkpoint eng`. Each one is a separate ~450 MB download fetched on first use.

### Python API

For more than one file, use a `Session` so the model loads once and is reused across calls:

```python
import eluate

with eluate.Session() as session:
    for path in ["a.mp4", "b.mp4", "c.mp4"]:
        session.elute(path)
```

`eluate.elute()`, `eluate.Session`, `eluate.Result`, and the `eluate.EluateError` exception hierarchy are the whole public surface. The API can also return the separated speech and sfx as WAV files for downstream tools (Whisper, loudness analysis, etc.) without re-extracting the audio:

```python
result = eluate.elute("interview.mp4", outputs=("speech", "sfx"))
# result.speech, result.sfx → WAV paths; result.video → None (not requested)
```

Full reference, including progress callbacks and the typed exceptions, is in [`docs/api.md`](docs/api.md).

## How it works

ELUATE extracts the audio with FFmpeg, runs it through the [Bandit v2](https://arxiv.org/abs/2407.07275) separator to split it into speech, music, and sfx, drops the music stem, mixes speech and sfx back together, and muxes that new audio track onto a straight copy of the original video.

```
 input.mp4 ─┬─▶ ffmpeg extract ─▶ 48 kHz WAV ─▶ Bandit v2
            │                                      │
            │                    ┌─────────────────┼─────────────────┐
            │                    ▼                 ▼                  ▼
            │                  speech            music               sfx
            │                    │               (drop)              │
            │                    └────────────── mix ────────────────┘
            │                                     │
            └─▶ ffmpeg mux ◀───────  speech + sfx audio track
                   │
                   ▼
           output_eluted.mp4   (video stream copied as-is)
```

The separator streams the audio through a fixed-size ring buffer instead of loading the whole track into memory, so peak memory stays roughly flat no matter how long the file is. On an 84-minute documentary on an M4 Mac with 24 GB, peak memory was about 19 GB versus about 41 GB for the unoptimized upstream reference (which then swapped ~20 GB to disk). Methodology and raw numbers are in [`docs/bench/`](docs/bench/).

## Known limitations

- It does well on clean mixes, however when music and speech or music and rythmic sfx (like train sound) overlap heavily you can get faint musical residue or a slight change to the voice. It's better to listen the result before relying on it.
- It is not faster than [Demucs](https://github.com/facebookresearch/demucs). I tuned the memory path for long files, not raw throughput, so on short clips Demucs will finish first. (Demucs is also aimed at music stem separation, which is a different job.)
- The CLI is video-in, video-out and never gives you stems. The Python API can export the speech and sfx WAVs, but neither surface ever exposes the music stem. ELUATE removes music; it does not hand it to you.
- There's a 6-hour duration cap and a disk-space precheck. Both are there to catch misdirected inputs; pass `--force` to override either.
- Only the default `multi` checkpoint has a recorded SHA-256. The other language checkpoints download without an integrity check and print a note when they do.
- I developed and tested this mostly on a Mac mini (Apple Silicon / MPS), with the CUDA path tuned for Colab. macOS and Linux are supported; Windows isn't tested.

## Files and privacy

Everything lives under your home directory:

| Path | What's there |
|---|---|
| `~/.eluate/models/` | Downloaded checkpoints (~450 MB each) and config |
| `~/Documents/ELUATE/` | Processed output videos |
| `~/.eluate/telemetry.jsonl` | Local debug log, only if you enable it |

Nothing is sent anywhere after the one-time model download. The debug log is off by default; set `ELUATE_TELEMETRY=1` to write a local JSONL record of processing stages, which helps when reporting a bug. It stays on your machine and contains nothing that identifies you or your files.

## License and credits

ELUATE's own code is MIT (see [`LICENSE`](LICENSE)). The Bandit v2 model weights are CC-BY-SA 4.0, from [Zenodo 12701995](https://zenodo.org/records/12701995). CC-BY-SA permits commercial use, but under share-alike terms that are operationally hostile to a lot of commercial software, so talk to a lawyer before shipping a commercial product built on ELUATE. This README is not legal advice.

Credits:

- [Karn N. Watcharasupat, Chih-Wei Wu, and Iroro Orife](https://arxiv.org/abs/2407.07275) for the Bandit v2 architecture and the Divide-and-Remaster v3 dataset.
- [ZFTurbo](https://github.com/ZFTurbo) for the [Music-Source-Separation-Training](https://github.com/ZFTurbo/Music-Source-Separation-Training) framework (MIT) that ELUATE's separator is built on.
- Built on PyTorch and torchaudio, librosa, Rich for the terminal UI, and FFmpeg.

It's a one-maintainer project plus AI. Fork it, improve it, open an issue or PR at <https://github.com/anaxoniclabs/ELUATE/issues>.
