Metadata-Version: 2.5
Name: vachaka
Version: 0.1.0
Summary: High-level Sanskrit recitation from text: Vāgdhenu chant and prose voices with line and word timings
Project-URL: Repository, https://github.com/vedicreader/vachaka
Project-URL: Documentation, https://vedicreader.github.io/vachaka/
Author-email: Karthik <karthik.rajgopal@hotmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: chant,nbdev,recitation,sanskrit,tts
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Requires-Python: <3.13,>=3.10
Requires-Dist: accelerate==0.34.2
Requires-Dist: cached-path
Requires-Dist: datasets
Requires-Dist: ema-pytorch>=0.5.2
Requires-Dist: fastcore>=1.8
Requires-Dist: huggingface-hub
Requires-Dist: indic-transliteration
Requires-Dist: jieba
Requires-Dist: librosa==0.11.0
Requires-Dist: matplotlib
Requires-Dist: numpy==1.26.4
Requires-Dist: pydub
Requires-Dist: pypinyin
Requires-Dist: safetensors
Requires-Dist: soundfile
Requires-Dist: torch==2.5.1
Requires-Dist: torchaudio==2.5.1
Requires-Dist: torchdiffeq
Requires-Dist: tqdm
Requires-Dist: transformers==4.46.3
Requires-Dist: vocos==0.1.0
Requires-Dist: wandb
Requires-Dist: x-transformers==2.19.7
Provides-Extra: check
Requires-Dist: nemo-toolkit[asr]>=2.1.0; extra == 'check'
Description-Content-Type: text/markdown

# vachaka


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

vachaka turns Devanagari into recited audio and tells you where every line and word falls in it. It is a thin layer over two models by
Prathosh A. P.: [Vāgdhenu](https://github.com/prathoshap/vagdhenu), a metre-aware chant TTS (an IndicF5 fine-tune with a BigVGAN-v2 vocoder), and
[Su-śrotā](https://huggingface.co/prathoshap/sushrota-sanskrit-asr), a Sanskrit ASR (IndicConformer fine-tuned on śāstric Sanskrit). Vāgdhenu does
the speaking. Su-śrotā listens back: it scores each render against its text, so a bad seed can be thrown away, and it force-aligns the words.

It was written for [VedicReader](https://vedicreader.com), which has thousands of Sanskrit verses and sūtras with no recording. The input there is a
title’s display lines, and the output has to be one audio file plus a `[start, end]` for every line and word, so the reader can follow along. That is
what `vachaka.recite` does.

## Listen

Clips from a pilot run over VedicReader’s texts. The voice is synthetic, and it is Vāgdhenu’s author’s. CER is Su-śrotā’s character error rate for
the clip against its text. The players work on the [docs site](https://vedicreader.github.io/vachaka/); GitHub and PyPI show only the text.

### Annapūrṇāṣṭakam, verse 2 (śārdūlavikrīḍita)

<audio controls preload="none" src="examples/stotra_shardula.mp3">

</audio>

नानारत्नविचित्रभूषणकरी हेमाम्बराडम्बरी<br>मुक्ताहारविलम्बमानविलसद्वक्षोजकुम्भान्तरी ।<br>काश्मीरागरुवासिताङ्गरुचिरे काशीपुराधीश्वरी<br>भिक्षां देहि कृपावलम्बनकरी मातान्नपूर्णेश्वरी ॥ २ ॥

chant, reference `shardulavikridita`, CER 0.031

### Bṛhat Parāśara Horā Śāstra 1.3 (anuṣṭubh)

<audio controls preload="none" src="examples/bphs_anushtubh.mp3">

</audio>

इति स्तुत्या सुसंहृष्टो मुनिस्तत्त्वविदां वरः।<br>अथादिदेश सच्छास्त्रं सारं यज्ज्योतिषां शुभम्॥ ३॥

chant, reference `anushtubh`, CER 0.0

### Jaimini Sūtras 1.1.1 (prose)

<audio controls preload="none" src="examples/jaimini_sutra.mp3">

</audio>

उपदेशं व्याख्यास्यामः॥

read, reference `anuṣṭubh`, CER 0.0

### One sūtra, two references

Nyāyasūtra 1.1.2 read twice. Vāgdhenu ships two prose references (`gadya`) and its author reads prose against the anuṣṭubh reference instead, finding
that gadya drops onsets. vachaka’s prose default follows the author. Judge for yourself:

anuṣṭubh reference <audio controls preload="none" src="examples/nyaya_prose_anushtubh_ref.mp3"></audio>

gadya reference <audio controls preload="none" src="examples/nyaya_prose_gadya_ref.mp3"></audio>

दुःखजन्मप्रवृत्तिदोषमिथ्याज्ञानानामुत्तरोत्तरापाये तदनन्तरापायादपवर्गः

## Numbers

Measured on an Apple M3 Max (36 GB), with the DiT on MPS and the vocoder on the CPU.

|  |  |
|----|----|
| loading Vāgdhenu | 10 to 17 s |
| rendering, warm | about 5.5 s per second of audio (a 9.5 s anuṣṭubh verse in 50 to 57 s) |
| share of render time in the DiT | 77% (the rest is BigVGAN on the CPU) |
| loading Su-śrotā | 9 s |
| transcribing a 10 s clip on the CPU | 0.4 s |
| alignment resolution | 40 ms (Su-śrotā’s frame) |

At that speed an hour of recitation takes five to six hours on a laptop. A CUDA GPU is much faster; Vāgdhenu’s author renders on one.

The pilot rendered 27 sections (293 s of audio) from a stotra, three jyotiṣa books, the Jaimini Sūtras and the Nyāyasūtra, taking the best of up to
three seeds per section:

| voice                                   | sections | mean CER | worst | exact |
|-----------------------------------------|----------|----------|-------|-------|
| chant, reference for the detected metre | 9        | 0.030    | 0.086 | 2     |
| prose, anuṣṭubh reference               | 14       | 0.016    | 0.118 | 9     |
| prose, gadya reference                  | 4        | 0.010    | 0.039 | 3     |

Two cautions. Su-śrotā comes from the same lab and its training data includes a TTS-speaker slice, so it may hear Vāgdhenu more kindly than a person
would; a low CER says the words are there, not that the chant is beautiful. And a section whose metre is not detected (āryā, a verse split over three
lines) is read as prose, which sounds flatter than a chant. Listen before you ship.

## Install

``` sh
pip install vachaka            # rendering
pip install 'vachaka[check]'   # Su-śrotā: transcripts, error rate, word timings
```

Python 3.10 to 3.12. The first [`Chanter()`](https://vedicreader.github.io/vachaka/core.html#chanter) downloads about 1.5 GB of weights from the Hugging Face Hub and fetches the sources of Vāgdhenu, BigVGAN and
IndicF5 at pinned commits into `~/.cache/vachaka` (set `VACHAKA_CACHE` to move it); none of the three is on PyPI, and the `f5-tts` that is differs from
IndicF5. Su-śrotā is 520 MB more. Writing mp3 needs `ffmpeg`.

## Use

Chant a verse given as its lines, and read prose:

``` python
from vachaka.core import Chanter
ch = Chanter()                        # cuda, then mps, then cpu
r = ch.chant(['वसुदेवसुतं देवं कंसचाणूरमर्दनम्', 'देवकीपरमानन्दं कृष्णं वन्दे जगद्गुरुम्'])
r.spans                               # [(0, 4230), (4980, 9550)]: one span per line, in ms
r.save('krishna.mp3')
ch.read('अथातो धर्मजिज्ञासा । चोदनालक्षणोऽर्थो धर्मः ।', ref='anuṣṭubh').save('mimamsa.mp3')
```

Check a render and time its words:

``` python
from vachaka.check import Ear, best_of
ear = Ear()
lines = ['वसुदेवसुतं देवं कंसचाणूरमर्दनम्', 'देवकीपरमानन्दं कृष्णं वन्दे जगद्गुरुम्']
r = best_of(lambda seed: ch.chant(lines, seed=seed), ear, ' '.join(lines))
r.cer                                 # 0.0
ear.hear(r.audio, r.sr)               # 'वसुदेवसुतं देवं कंसचाणूरमर्दनं देवकीपरमानन्दं कृष्णं वन्दे जगद्गुरुं'
ear.words(r, [l.split() for l in lines])
# [[['वसुदेवसुतं', 120, 1279], ['देवं', 1279, 1878], ['कंसचाणूरमर्दनम्', 1878, 3756]],
#  [['देवकीपरमानन्दं', 5100, 7018], ['कृष्णं', 7018, 7697], ['वन्दे', 7697, 8217], ['जगद्गुरुम्', 8217, 9056]]]
```

Recite whole sections from their display lines, with daṇḍas, verse numbers and accents left in; each line comes back with its span and words:

``` python
from vachaka.recite import recite
rec, info = recite(ch, ear, [dict(lines=['नित्यानन्दकरी वराभयकरी सौन्दर्यरत्नाकरी', 'निर्धूताखिलघोरपावनकरी प्रत्यक्षमाहेश्वरी ।',
                                         'प्रालेयाचलवंशपावनकरी काशीपुराधीश्वरी', 'भिक्षां देहि कृपावलम्बनकरी मातान्नपूर्णेश्वरी ॥ १ ॥'])])
info[0]['kind'], info[0]['ref'], info[0]['lines'][0]   # 'chant', 'shardulavikridita', {'s': ..., 'e': ..., 'w': [...]}
```

For batches there is a resumable CLI over jsonl: `vachaka-recite jobs.jsonl results.jsonl`, where a job is `dict(id, out, sections)`.

## How it works

[`recite`](https://vedicreader.github.io/vachaka/recite.html#recite) groups display lines into pieces, closing a piece after a daṇḍa, so the two pādas of a hemistich printed on two lines render together, as
Vāgdhenu expects. A section whose text scans as a known metre is chanted against the reference clip for that metre; anything else is read in chunks
of at most 24 akṣaras. Vāgdhenu renders each piece separately and joins them with short silences; vachaka records where each piece landed. Su-śrotā
then transcribes the whole render, and if the error rate is above 8% the section is rendered again with another seed, keeping the best of three.
Finally the words of each piece are force-aligned (torchaudio’s CTC aligner on Su-śrotā’s posteriors) inside that piece’s span, so one bad line can’t
pull the rest out of place.

Two things had to be patched to run this on a Mac. BigVGAN’s transposed convolution has more than 65,536 output channels, which MPS rejects, so the
vocoder runs on the CPU. And Su-śrotā’s checkpoint names AI4Bharat’s NeMo fork; vachaka rewrites its config for upstream NeMo (the tokenizer type
becomes `agg`, the fork’s `multisoftmax` keys go) and uses only the encoder and the CTC head, the Sanskrit columns 4096 to 4351.

## Limits

No Vedic svaras: Vāgdhenu is laukika chant, and accents are stripped before rendering. One voice, the model author’s. The reference clip sets pace and
melody but not timbre. Metres outside Vāgdhenu’s bank fall back to vasantatilakā when detected as some other vṛtta, and to prose when not detected at
all. Prose sounds like careful reading, not chant.

## Credits and licences

vachaka is Apache-2.0. Vāgdhenu (code Apache-2.0; weights redistributed under IndicF5’s MIT and NVIDIA BigVGAN’s licence) and Su-śrotā (fine-tuned from
AI4Bharat’s IndicConformer, MIT) are by Prathosh A. P.; please cite Vāgdhenu if you use its audio. The model card asks that the voice not be used to
impersonate anyone. Say that the audio is synthetic wherever you publish it.
