Metadata-Version: 2.5
Name: index-tts-2.5-mlx
Version: 0.1.0
Summary: IndexTTS-2.5 voice cloning on Apple Silicon: MLX int8, torch-free, one-click uvx
Project-URL: Homepage, https://huggingface.co/yunfengwang/IndexTTS-2.5-mlx
Project-URL: Source, https://huggingface.co/yunfengwang/IndexTTS-2.5-mlx
Author: yunfengwang
License: Bilibili-IndexTTS
Keywords: apple-silicon,indextts,mlx,tts,voice-cloning
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.24
Requires-Dist: kaldi-native-fbank>=1.18
Requires-Dist: librosa>=0.10
Requires-Dist: mlx>=0.24
Requires-Dist: numpy>=1.24
Requires-Dist: pyyaml>=6.0
Requires-Dist: soundfile>=0.12
Requires-Dist: tiktoken>=0.7
Requires-Dist: transformers>=4.45
Requires-Dist: wetext
Description-Content-Type: text/markdown

# index-tts-2.5-mlx

[IndexTTS-2.5](https://modelscope.cn/models/IndexTeam/IndexTTS-2.5) voice cloning on **Apple Silicon**, rebuilt on **MLX** with an **int8-quantized GPT** decoder. Torch-free, runs entirely on the GPU, and ships as a one-click `uvx` package that auto-downloads the weights from Hugging Face.

- **Voice cloning** from a short reference clip (≤15 s) — Chinese / English / Japanese / Cantonese, including mixed-language text.
- **Faster than real-time** (RTF ≈ 0.45) and **~2.4× faster than the official PyTorch MPS backend**.
- **Torch-free**: numpy + MLX only. No PyTorch, no ONNX Runtime.

> Hardware: Apple Silicon Mac (M1 or newer), macOS 13+, Python 3.10+. MLX uses the unified-memory GPU.

---

## Install / one-click run

No install needed with [uv](https://docs.astral.sh/uv/):

```bash
uvx index-tts-2.5-mlx synth \
    --ref /path/to/voice.wav \
    --text "大家好, this is IndexTTS running on MLX." \
    --out out.wav
```

The first run downloads the int8 model (~5 GB) from Hugging Face into the standard HF cache (`~/.cache/huggingface`); later runs reuse it. Each `synth` prints a timing report:

```
wrote out.wav
audio       2.83 s
load        2.12 s   (model download + weight load)
clone       1.69 s   (speaker embedding from --ref)
synth       1.32 s
RTF        0.467     (2.14x realtime; <1 = faster than real-time)
stages    gpt=0.35s  codec=0.00s  regulator=0.00s  cfm=0.46s  bigvgan=0.51s
```

(`load` is only slow the very first time, while it downloads. `synth` is the marginal cost per line once warm — reuse one `spk` across lines to skip repeated `clone` work.)

Or install into an environment:

```bash
pip install index-tts-2.5-mlx
```

Pre-download the weights ahead of time:

```bash
uvx index-tts-2.5-mlx download
```

## CLI usage

```
index-tts-2.5-mlx synth --ref REF.wav --text "..." --out out.wav [options]
```

| Option | Default | Description |
|---|---|---|
| `--ref` | (required) | Reference audio to clone (≤15 s, clear speech). |
| `--text` | (required) | Text to synthesize (zh/en/ja/yue, mixed OK). |
| `--out` | `output.wav` | Output WAV path (22050 Hz, int16). |
| `--lang` | `zh` | Language hint: `zh`, `en`, `ja`, `yue`. |
| `--greedy` | off | Greedy decoding (deterministic). |
| `--seed` | random | RNG seed for sampling. |
| `--top-k` / `--top-p` / `--temperature` | `30` / `0.8` / `0.8` | Sampling controls. |
| `--repetition-penalty` | `10.0` | Repetition penalty. |
| `--max-mel-tokens` | `1500` | Max acoustic tokens per segment. |
| `--duration-factor` | `1.0` | Speech-rate multiplier. |
| `--n-timesteps` / `--cfg-rate` | `25` / `0.7` | Flow-matching solver controls. |
| `--model-dir` | auto | Use a local weight dir instead of downloading. |
| `--no-normalization` | off | Disable text normalization. |

## Python API

```python
from index_tts_2_5_mlx import IndexTTS

tts = IndexTTS()                       # auto-downloads weights on first use
sr, pcm = tts.clone(
    "AI 模型在 2025 年处理了 100 万条数据。",
    ref_audio_path="voice.wav",
    out="clone.wav",                   # optional; also returns pcm
    lang="zh",
)

# Reuse one cloned voice across many lines (build the speaker once):
spk = tts.build_speaker("voice.wav")
for i, line in enumerate(["第一句。", "Second sentence.", "第三句。"]):
    sr, pcm = tts.clone(line, ref_audio_path=None, spk=spk, out=f"line{i}.wav")
```

`synthesize(...)` returns the raw int16 PCM array (`numpy`) at `tts.sample_rate` (22050 Hz); `clone(...)` additionally writes a WAV when `out` is given. Use your own reference audio only with permission — see **License**.

## Speed

End-to-end synthesis, warm, mean of 3 runs (Apple M5 Pro). **RTF** = synthesis time ÷ audio duration (lower is better; **<1 = faster than real-time**).

| Backend | fx0 RTF | fx1 RTF | vs PyTorch MPS |
|---|---|---|---|
| PyTorch MPS (official) | 1.17 | 1.11 | 1.0× |
| **MLX fp32** | 0.67 | 0.71 | ~1.7× |
| **MLX int8 (this package)** | **0.47** | **0.45** | **~2.4×** |

Stage breakdown (int8, ~3 s of audio): GPT decode ≈ 0.35 s, flow-matching CFM ≈ 0.48 s, BigVGAN vocoder ≈ 0.56 s. The int8 quantization fuses dequant into the Metal matmul kernels, which is where the GPT autoregressive decode speedup comes from; the other modules are compute-bound and stay fp32.

## Quality / effect

**Numeric fidelity** — each MLX module matches the PyTorch reference with cosine similarity ≥ 0.999; the full greedy pipeline reproduces the reference acoustic tokens exactly (fx0 73/73, fx1 83/83). Vocoder output matches the reference at mel-spectrogram SNR ≥ 25 dB (inaudible difference).

**Voice cloning** — measured with a CampPlus speaker-embedding cosine between each synthesized clip and its reference vs. an unrelated voice. Every clip scores higher against its own reference, confirming the timbre follows the given reference:

| Clip | sim(own ref) | sim(other voice) | follows ref |
|---|---|---|---|
| clone A ×3 | 0.61–0.73 | 0.29–0.39 | ✓ |
| clone B ×3 | 0.51–0.61 | 0.48–0.50 | ✓ |

**Intelligibility (ASR, Whisper)** — synthesized mixed-language clips with numbers and abbreviations transcribe correctly, e.g. `AI 模型在 2025 年处理了 100 万条数据。` → “AI…2025 年处理了 100 万条数据”, and `Use the CPU or GPU, 都可以` → “用 CPU 或 GPU 都可以”. Cloning quality tracks reference quality: use a clean, natural recording.

## How it works

Pipeline: text frontend (tiktoken + wetext normalization) → w2v-bert semantic features → semantic codec → **int8 GPT** autoregressive acoustic tokens → length regulator → flow-matching **CFM** (DiT) → **BigVGAN** vocoder → 22050 Hz WAV. The orchestration (samplers, solvers, DSP) is numpy; the heavy networks are MLX modules loading the shipped safetensors.

Model weights: [`yunfengwang/IndexTTS-2.5-mlx`](https://huggingface.co/yunfengwang/IndexTTS-2.5-mlx) (int8 GPT + fp32 feed-forward modules).

## License

Code in this package is provided under the same terms as the upstream project. IndexTTS-2.5 model weights are subject to the original **Bilibili IndexTTS** license — see the upstream model card. Use voice cloning responsibly and only with consent from the voice owner.
