Metadata-Version: 2.5
Name: index-tts-2.5-mnn
Version: 0.1.0
Summary: IndexTTS-2.5 voice cloning on MNN: fast CPU inference, torch-free, one-click uvx
Project-URL: Homepage, https://huggingface.co/yunfengwang/IndexTTS-2.5-mnn
Project-URL: Source, https://huggingface.co/yunfengwang/IndexTTS-2.5-mnn
Author: yunfengwang
License: Bilibili-IndexTTS
Keywords: cpu,indextts,mnn,tts,voice-cloning
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.24
Requires-Dist: kaldi-native-fbank>=1.18
Requires-Dist: librosa>=0.10
Requires-Dist: mnn>=2.6
Requires-Dist: numpy>=1.24
Requires-Dist: pyyaml>=6.0
Requires-Dist: soundfile>=0.12
Requires-Dist: tiktoken>=0.7
Requires-Dist: transformers>=4.45
Requires-Dist: wetext
Description-Content-Type: text/markdown

# index-tts-2.5-mnn

[IndexTTS-2.5](https://modelscope.cn/models/IndexTeam/IndexTTS-2.5) voice cloning on **MNN** — fast **CPU** inference (x86 / ARM, Linux / Windows / macOS), torch-free, and ships as a one-click `uvx` package that auto-downloads the weights from Hugging Face.

- **Voice cloning** from a short reference clip (≤15 s) — Chinese / English / Japanese / Cantonese, including mixed-language text with numbers and abbreviations.
- **Bit-exact output**: both shipped quant sets reproduce the PyTorch CPU reference greedy acoustic tokens exactly (73/73 and 83/83 on the verification fixtures); every stage matches at cosine ≥ 0.9999.
- **Torch-free**: numpy + pymnn only. No PyTorch, no ONNX Runtime, no MLX.
- **Quantized by default**: the `mixed` set ships **fp16 GPT weights** (fp32 compute) — 2 GB less download than fp32 with identical output.

> Platform: any OS with Python 3.10+. The pip `MNN` wheel is CPU-only; on Apple Silicon the [MLX build](https://pypi.org/project/index-tts-2.5-mlx/) (GPU) is ~7× faster than real-time, and on NVIDIA GPUs the [ONNX build](https://pypi.org/project/index-tts-2.5-onnx/) with CUDA is the fast path. This package is the fast **CPU** option — its BigVGAN vocoder runs ~4× faster than ONNX Runtime CPU.

---

## Features

- **Zero-shot voice cloning** — supply any ≤15 s clean reference; the timbre and speaking style are carried into the output. Build the speaker once and reuse it across unlimited lines.
- **Multilingual + code-switching** — `zh`, `en`, `ja`, `yue`, and mixed text in a single sentence (e.g. `Use the CPU or GPU, 都可以`).
- **Text normalization** — numbers, abbreviations and symbols are read out correctly via `wetext`. Disable with `--no-normalization`.
- **Rich decoding controls** — greedy or sampling (`top_k` / `top_p` / `temperature` / `seed`), `repetition_penalty`, `duration_factor` (speech rate), and the flow-matching solver knobs (`n_timesteps`, `cfg_rate`).
- **Auto-download** — weights pull from Hugging Face on first run and are cached for reuse; CLI and Python API share the same cache.
- **Timing report** — every `synth` prints load / clone / synth time, RTF, and a per-stage breakdown.

## Install / one-click run

No install needed with [uv](https://docs.astral.sh/uv/):

```bash
uvx index-tts-2.5-mnn synth \
    --ref /path/to/voice.wav \
    --text "大家好, this is IndexTTS running on MNN." \
    --out out.wav
```

The first run downloads the `mixed` weights (~5 GB) from Hugging Face into the standard HF cache; later runs reuse it. Use `--quant fp32` for the all-fp32 set (~7 GB, marginally faster GPT decode).

Or install into an environment:

```bash
pip install index-tts-2.5-mnn
```

Pre-download the weights ahead of time:

```bash
uvx index-tts-2.5-mnn download            # mixed (default)
uvx index-tts-2.5-mnn download --quant fp32
```

## CLI usage

```
index-tts-2.5-mnn synth --ref REF.wav --text "..." --out out.wav [options]
```

| Option | Default | Description |
|---|---|---|
| `--ref` | (required) | Reference audio to clone (≤15 s, clear speech). |
| `--text` | (required) | Text to synthesize (zh/en/ja/yue, mixed OK). |
| `--out` | `output.wav` | Output WAV path (22050 Hz, int16). |
| `--lang` | `zh` | Language hint: `zh`, `en`, `ja`, `yue`. |
| `--quant` | `mixed` | `mixed` (fp16 GPT weights + fp32 rest) or `fp32`. |
| `--threads` | `4` | CPU threads. |
| `--greedy` | off | Greedy decoding (deterministic). |
| `--seed` | random | RNG seed for sampling. |
| `--top-k` / `--top-p` / `--temperature` | `30` / `0.8` / `0.8` | Sampling controls. |
| `--repetition-penalty` | `10.0` | Repetition penalty. |
| `--max-mel-tokens` | `1500` | Max acoustic tokens per segment. |
| `--duration-factor` | `1.0` | Speech-rate multiplier. |
| `--n-timesteps` / `--cfg-rate` | `25` / `0.7` | Flow-matching solver controls. |
| `--model-dir` | auto | Use a local weight dir instead of downloading. |
| `--no-normalization` | off | Disable text normalization. |

## Python API

```python
from index_tts_2_5_mnn import IndexTTS

tts = IndexTTS(quant="mixed")          # auto-downloads weights on first use
sr, pcm = tts.clone(
    "AI 模型在 2025 年处理了 100 万条数据。",
    ref_audio_path="voice.wav",
    out="clone.wav",                   # optional; also returns pcm
    lang="zh",
)

# Reuse one cloned voice across many lines (build the speaker once):
spk = tts.build_speaker("voice.wav")
for i, line in enumerate(["第一句。", "Second sentence.", "第三句。"]):
    sr, pcm = tts.clone(line, ref_audio_path=None, spk=spk, out=f"line{i}.wav")
```

`synthesize(...)` returns the raw int16 PCM array (`numpy`) at `tts.sample_rate` (22050 Hz); `clone(...)` additionally writes a WAV when `out` is given. Use your own reference audio only with permission — see **License**.

## Speed

End-to-end synthesis, warm, 4 threads (Apple M5 Pro, ~3 s of audio). **RTF** = synthesis time ÷ audio duration (lower is better).

| Backend | synth | RTF | gpt | cfm | bigvgan |
|---|---|---|---|---|---|
| MNN fp32 | 9.20 s | 3.16 | 3.61 s | 4.42 s | 1.12 s |
| MNN mixed (default) | 9.48 s | 3.25 | 3.84 s | 4.42 s | 1.12 s |
| ONNX Runtime fp32 (CPU) | 9.53 s | 3.27 | 1.20 s | 3.51 s | 4.81 s |

MNN's vocoder is ~4× faster than ONNX Runtime CPU (1.1 s vs 4.8 s); ORT wins the GPT decode. End-to-end they land at the same place on this machine — pick MNN for the smaller quantized download and the self-contained CPU wheel.

## Quality / effect

**Numeric fidelity** — fp32 and mixed both reproduce the PyTorch CPU reference greedy acoustic tokens **exactly** (fx0 73/73, fx1 83/83); stage outputs match at cosine ≥ 0.9999; vocoder output matches at mel-spectrogram SNR ≈ 70 dB (inaudible).

**Quantization: what ships and why.** The GPT is 60% of the model size and its fp16 *weights* (dequantized to fp32 at load, fp32 compute) produce bit-identical greedy decodes — that's the `mixed` set. Everything deeper was measured and rejected:

- **int8/int4 BigVGAN** destroys the vocoder (SNR ≤ 15 dB vs 64.5 dB fp32; int4 is pure noise at −1.4 dB).
- **int8 CFM** mel outliers (maxdiff 0.6–1.2) collapse even an fp32 vocoder to 7 dB SNR; **fp16 CFM** reaches only 18 dB.
- **int8 CAMPPlus / semantic encoder** shifts the GPT conditioning just enough (cos 0.992–0.999) to flip greedy argmax — the EN fixture keeps only 17% of reference tokens.
- **int8 GPT** (any block config) flips a close argmax (35% match); **fp16 GPT compute** (MNN `precision="low"`) also flips it (≤7%) — fp16 is only safe as a *weight* format.

So: fp16 GPT weights in, everything else fp32. No int8/int4 set is shipped because none preserved quality.

## How it works

Pipeline: text frontend (tiktoken + wetext normalization) → w2v-bert semantic features → semantic codec → **GPT** autoregressive acoustic tokens → length regulator → flow-matching **CFM** (DiT) → **BigVGAN** vocoder → 22050 Hz WAV. The orchestration (samplers, solvers, DSP) is numpy; the heavy networks are MNN graphs converted from the ONNX export and verified stage-by-stage against the PyTorch reference.

Model weights: [`yunfengwang/IndexTTS-2.5-mnn`](https://huggingface.co/yunfengwang/IndexTTS-2.5-mnn).

## License

Code in this package is provided under the same terms as the upstream project. IndexTTS-2.5 model weights are subject to the original **Bilibili IndexTTS** license — see the upstream model card. Use voice cloning responsibly and only with consent from the voice owner.
