Metadata-Version: 2.4
Name: haaztest-kitten-tts-cpp
Version: 0.2.0.post3
Summary: CPU KittenTTS2 inference with native C++ bindings
Home-page: https://github.com/KittenML/kitten-tts-2-cpp
Author: KittenML
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: AUTHORS
Requires-Dist: numpy>=1.26
Requires-Dist: torch==2.13.0
Requires-Dist: soundfile>=0.12
Requires-Dist: certifi>=2024.7.4
Requires-Dist: huggingface_hub<2,>=0.25
Requires-Dist: safetensors>=0.4
Requires-Dist: gguf>=0.19
Requires-Dist: phonemizer>=3.3
Requires-Dist: espeakng_loader>=0.2.4
Requires-Dist: misaki==0.9.4; python_version < "3.13"
Requires-Dist: spacy<4,>=3.8; python_version < "3.13"
Requires-Dist: num2words>=0.5.13; python_version < "3.13"
Dynamic: author
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# KittenTTS2 C++

C++ inference for KittenTTS2 on the CPU, with an optional NVIDIA GPU (CUDA) path, built-in voices, expression controls, and prepared reference voices. The runtime downloads prepared model assets and runs synthesis natively in C++. Users do not need to run the Python exporter.

The speech language model runs through llama.cpp/GGML, text normalization uses the [kitten-text-processing](https://github.com/KittenML/kitten-text-processing) submodule, and the S3 waveform decoder uses LibTorch on the CPU or on the GPU.

The same package also runs the small multilingual models Kitten TTS 2 mini, nano and pico (`KittenTTS2Tiny`, all in ggml, no LibTorch for synthesis); see [Kitten TTS 2 mini, nano and pico](#kitten-tts-2-mini-nano-and-pico) below.

## Features

- Built-in voices, including Bruno, Bella, Jasper, Luna, Rosie, Hugo, Kiki, and Leo.
- Emotion tags, vocal events, and emphasis: `[joyful]`, `<laugh>`, and `(((really)))`.
- Stable and expressive sampling presets, sentence-aware chunking, and audio joins.
- A 1.03 GB GGUF with lossless packing of the checkpoint's ternary weights, plus an FP16 reference export.
- Default S3 and optional `student_w4` / `student_w8` decoder exports.
- Mono float32 WAV output at 24 kHz.

## Python interface

Precompiled wheel builds are configured for Linux x86-64/ARM64, Apple Silicon macOS, Windows x86-64, and experimental Windows ARM64. See the [wheel matrix and installation guide](packaging/README.md).

The `py-interface` branch includes a native Python package with a KittenTTS-style API. `setup.py` invokes CMake during installation and bundles the normalizer grammar data. Install a CPU PyTorch build first, then build against that same installation:

```sh
git submodule update --init vendor/kitten-text-processing
python3 -m venv .venv
. .venv/bin/activate
python -m pip install torch==2.13.0 --index-url https://download.pytorch.org/whl/cpu
python -m pip install setuptools wheel cmake numpy soundfile
python -m pip install . --no-build-isolation
```

A C++20 compiler and OpenSSL development headers are required for HTTPS downloads (`libssl-dev` on Debian/Ubuntu). Set `CMAKE_BUILD_PARALLEL_LEVEL=8` to control build parallelism. `CMAKE_ARGS` accepts extra CMake flags, such as `-DOPENSSL_ROOT_DIR=/path/to/openssl`. Builds optimize for the local CPU; use `CMAKE_ARGS="-DGGML_NATIVE=OFF"` when building a wheel for other machines. Source-built wheels require the exact PyTorch release used at build time and the linked system OpenSSL libraries. The portable CI wheels instead link LibreSSL statically.

```python
from kitten_tts_cpp import KittenTTS  # KittenTTS2 and get_model are also available

with KittenTTS("KittenML/kitten-tts-2", decoder="student_w4", threads=8) as model:
    print(model.available_voices)
    audio = model.generate("Hello again.", voice="Bruno", advanced={"seed": 1234})
    model.generate_to_file("Hello again.", "hello.wav", voice="Bruno")
    for chunk in model.generate_stream("A longer passage goes here.", voice="Bruno"):
        pass  # Each chunk is a float32 NumPy array at model.sample_rate.
```

For NVIDIA GPUs, install the GPU package instead (`kitten-tts-cpp-cu12` on Linux, `kitten-tts-cpp-cu13` on Windows) and pass `gpu_layers=99`; see the [GPU wheel guide](packaging/README.md#using-the-gpu).

Weights and the decoder stay loaded between calls. The GIL is released during native loading and inference. Native inference calls are serialized because the TorchScript decoder uses process-wide random and thread settings. Streaming yields untrimmed sentence chunks as they are decoded; close the iterator when stopping early. `generate` applies the same chunk joins as the CLI. `last_report` contains token IDs, generation timings and the LM placement (`last_report["lm"]["placement"]`).

The constructor also accepts `cache_dir`, `hf_token`, `revision`, `offline`, `decoder_threads`, `repack`, `model_path` (a local GGUF override), `data_dir`, and `ca_file` (a custom HTTPS trust bundle). A local prepared asset directory can replace the repository ID. Generation supports the Python KittenTTS2 preset, sampling, normalization, reference-prompt, and `advanced` options, including `use_emotion`.

Only KittenTTS2 and `weights="packed"` are supported; `device=` accepts only `"cpu"` (use `gpu_layers` and `decoder_device` for the GPU). This package uses the separate `kitten_tts_cpp` import name so it can coexist with `kittenml`. Raw-audio `reference=` cloning is not implemented; prepare a reference with `prepare_reference.py` and select its voice name. It does not provide the older ONNX models or guarantee identical sampled output to Python's original PyTorch backend.

### Kitten TTS 2 mini, nano and pico

```sh
python3 -m venv .venv && . .venv/bin/activate
python -m pip install -U pip
pip install kitten-tts-cpp --extra-index-url https://download.pytorch.org/whl/cpu
python -c "from kitten_tts_cpp import KittenTTS2Tiny; KittenTTS2Tiny('KittenML/kitten-tts-2-nano').generate_to_file('Guten Morgen.', 'hallo.wav', language='de')"
```

[KittenML/kitten-tts-2-mini](https://huggingface.co/KittenML/kitten-tts-2-mini) speaks 16 languages,
[-nano](https://huggingface.co/KittenML/kitten-tts-2-nano) and [-pico](https://huggingface.co/KittenML/kitten-tts-2-pico)
8 each. The first use downloads the official weights and converts them to GGUF once. English needs Python 3.10 to
3.12 (its text front end, misaki 0.9.4 and spaCy, installs only there); on newer Python the other languages work and
English raises an error that says so. Options, weights, speed and the C API:
[tools/kitten-tts-tiny/README.md](tools/kitten-tts-tiny/README.md).

For development, `python setup.py build_ext --inplace` builds the extension; set `PYTHONPATH=python` to import it directly. `python setup.py bdist_wheel` produces an installable wheel, and `python setup.py sdist` includes the native sources and normalizer submodule contents.

## Build

The build commands use a Linux shell. Requires a C++20 compiler, CMake, Python 3.10+, OpenSSL development files (`libssl-dev` on Debian/Ubuntu), and LibTorch 2.13.0. Use the CPU PyTorch environment from [Python interface](#python-interface); it supplies LibTorch. Publishers use a separate [export environment](tools/kitten-tts/README.md#export-assets-publishers-only).

```sh
git submodule update --init vendor/kitten-text-processing

cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_STANDARD=20 \
  -DCMAKE_CXX_STANDARD_REQUIRED=ON \
  -DGGML_CUDA=OFF -DGGML_METAL=OFF \
  -DLLAMA_BUILD_KITTEN_TTS=ON \
  -DLLAMA_BUILD_SERVER=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_EXAMPLES=OFF \
  -DCMAKE_PREFIX_PATH="$(python -c 'import torch; print(torch.utils.cmake_prefix_path)')"

cmake --build build --target kitten-tts -j 8
```

The normalizer prepares its grammar data using Python at build time. To use existing grammar data, configure with `-DKITTEN_DATA_DIR=/path/to/data`.

## GPU (CUDA) build

For the recommended full GPU setup, install an NVIDIA driver, CUDA 12.8 (`nvcc` on `PATH`), and CMake 3.28+. From the repository root:

```sh
python3 -m venv .venv-cuda
. .venv-cuda/bin/activate
python -m pip install torch==2.13.0 --index-url https://download.pytorch.org/whl/cu129
git submodule update --init vendor/kitten-text-processing
cmake -S . -B build-cuda \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_STANDARD=20 \
  -DCMAKE_CXX_STANDARD_REQUIRED=ON \
  -DGGML_CUDA=ON -DGGML_METAL=OFF \
  -DCMAKE_CUDA_ARCHITECTURES='75-real;86-real;89-real;120a-real' \
  -DLLAMA_BUILD_KITTEN_TTS=ON \
  -DLLAMA_BUILD_SERVER=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_EXAMPLES=OFF \
  -DCMAKE_PREFIX_PATH="$(python -c 'import torch; print(torch.utils.cmake_prefix_path)')"
cmake --build build-cuda --target kitten-tts -j 8
build-cuda/bin/kitten-tts --gpu-layers 99 --decoder-device cuda --timings --text 'Hello from the GPU.' --output gpu.wav
```

`--gpu-layers 99` runs the language model on the GPU; the default, 0, keeps the CPU path and its output unchanged. To run the S3 decoder on the GPU as well, add `--decoder-device cuda`: the runtime downloads the decoder's CUDA export (`default` and `student_w4`; `student_w8` has none) and needs a CUDA build of LibTorch ([how](tools/kitten-tts/README.md#cuda-waveform-decoder)).

| Setup (RTX 4090) | Real-time factor (lower is faster) | GPU memory |
|---|---:|---:|
| Language model and decoder on the GPU (recommended) | 0.15 | about 2.2-2.9 GB |
| Language model on the GPU, decoder on the CPU | 0.41 | about 1.7 GB |
| CPU only | 1.28 | none |

Running the decoder on the CPU, or only part of the language model on the GPU (`--allow-hybrid`), is supported but slower. Details: [GPU guide](tools/kitten-tts/README.md#gpu-cuda-build).

## Download and run

The runtime fetches `config.json` from `KittenML/kitten-tts-2` first, then downloads its GGUF, selected TorchScript decoder, and matching voice conditioning. Assets are cached under `$XDG_CACHE_HOME/kitten-tts`, or `~/.cache/kitten-tts` when it is unset. `HOME` falls back to `USERPROFILE` on Windows.

```sh
build/bin/kitten-tts --text 'Hello from Kitten T T S.' --output hello.wav
build/bin/kitten-tts --decoder student_w4 --text 'Hello again.' --output w4.wav
```

Use `--repo OWNER/REPO` to select another KittenTTS2 repository, `--revision COMMIT` to pin a model version, `--cache-dir DIR` to change the cache location, and `--offline` to use cached files without network access. `HF_TOKEN` supports private repositories. `--download-only --decoder student_w4` prepares the cache without synthesis. Only assets needed for the selected mode are downloaded.

The model repository must publish a `cpp` manifest and prepared files as described in the [publisher guide](tools/kitten-tts/README.md#publish-native-assets). Local exports and model repository directories remain supported through `--assets DIR`; `--model FILE` overrides the GGUF.

The default GGUF uses this fork's `TQ2_1` format: 256 ternary weights packed into 64 bytes, plus two FP16 scales for the checkpoint's 128-weight groups. The exporter requires exact reconstruction. This reduces the model from 1.45 GB with Q4_0 to 1.03 GB. Embeddings remain FP16 and one-dimensional tensors remain FP32. GGML's quantized activation arithmetic can still introduce small logit differences.

`TQ2_1` requires this fork; it is not standard `TQ2_0` and stock llama.cpp cannot load it. The fork has CPU kernels for it, and GPU builds repack it losslessly to Q2_0 at load time. Use `--outtype ternary-q4_0` for the previous standard Q4_0 storage format, or `--outtype f16` for the reference export.

On supported Intel CPUs, the runtime losslessly repacks TQ2_1 weights into the existing AMX Q4_0 layout at load time. The GGUF stays 1.03 GB, but runtime transformer-weight storage grows from about 357 MiB to 756 MiB. Use `--no-repack` to keep the compact CPU representation; it affects only CPU buffers, not the GPU repack to Q2_0. CPUs without AMX use the existing compact kernels.

See the [detailed guide](tools/kitten-tts/README.md) for FP16 exports, student decoders, reference voice preparation, and the normalizer submodule setup.

## Generate speech

```sh
build/bin/kitten-tts \
  --text 'Hello there. This is Kitten T T S running on the CPU.' \
  --voice Bruno --threads 8 --seed 1234 \
  --output hello.wav --report hello.json
```

For expressive speech:

```sh
build/bin/kitten-tts \
  --text '[joyful] We won the grant <laugh> I can (((hardly))) believe it!' \
  --voice Kiki --preset expressive \
  --output expressive.wav
```

Use `--help` for sampling, repetition penalties, chunking, and decoder thread settings. `--tokens-only` skips waveform decoding; `--repeat 3` benchmarks repeated synthesis with the model and decoder kept loaded.

## Python parity and CPU performance

The implementation is checked against `kittenml.kittentts2`:

| Check | Recorded result |
| --- | --- |
| Text normalization and chunking | 228 cases passed |
| Sampling rules | 12 cases passed |
| Audio joins | 3 cases passed |
| Packed weights | All 310 tensors reconstruct the FP16 reference exactly |
| Native waveform decoding | Agreement with Python within numerical tolerances in tested cases |
| Packed LM logits | Close numerical agreement with the Python reference |

On a Xeon Platinum 8462Y+ with eight pinned CPU cores, the AMX path reduced a 256-token forced-sequence workload from **6.40 to 5.60 seconds** (about **14% higher throughput**). Prompt evaluation fell from **1.09 to 0.58 seconds**. TQ2_1 reached **45.7 generated tokens/second**, comparable to Q4_0 at **44.3** in the same test. These are medians of three warmed runs, including prompt evaluation and sampling but excluding model loading. Performance varies with hardware and workload.

Run the differential checks with:

```sh
python tools/kitten-tts/check_parity.py \
  --binary build/bin/kitten-tts \
  --python-package ../KittenTTS \
  --repo ../kitten-tts-2 --assets models/kitten2 \
  --model models/kitten2/model-tq2_1.gguf \
  --lm --decoder
```

Measurements and source revisions are recorded in [validation.json](tools/kitten-tts/validation.json).

## Current scope

Sampled output is not bit-identical to Python: BF16 arithmetic, GGML activation quantization, and sampling RNGs differ. Matching seeds do not guarantee matching generated tokens across runtimes.

Voice cloning requires an offline Python preparation step and a reference transcript. The Python interface supports sentence-chunk streaming; sample-by-sample streaming is not implemented. Waveform decoding currently depends on LibTorch, on the CPU or on CUDA.

## Credits

KittenTTS2 and kitten-text-processing are from KittenML. See the [implementation guide](tools/kitten-tts/README.md#attribution) for decoder attribution and license details.

Built on [DeepGrove's llama.cpp fork](https://github.com/deepgrove-ai/llama.cpp).
