vachaka

vachaka turns Devanagari into recited audio and tells you where every line and word falls in it. It is a thin layer over two models by Prathosh A. P.: Vāgdhenu, a metre-aware chant TTS (an IndicF5 fine-tune with a BigVGAN-v2 vocoder), and Su-śrotā, a Sanskrit ASR (IndicConformer fine-tuned on śāstric Sanskrit). Vāgdhenu does the speaking. Su-śrotā listens back: it scores each render against its text, so a bad seed can be thrown away, and it force-aligns the words.

It was written for VedicReader, which has thousands of Sanskrit verses and sūtras with no recording. The input there is a title’s display lines, and the output has to be one audio file plus a [start, end] for every line and word, so the reader can follow along. That is what vachaka.recite does.

Listen

Clips from a pilot run over VedicReader’s texts. The voice is synthetic, and it is Vāgdhenu’s author’s. CER is Su-śrotā’s character error rate for the clip against its text. The players work on the docs site; GitHub and PyPI show only the text.

Annapūrṇāṣṭakam, verse 2 (śārdūlavikrīḍita)

नानारत्नविचित्रभूषणकरी हेमाम्बराडम्बरी
मुक्ताहारविलम्बमानविलसद्वक्षोजकुम्भान्तरी ।
काश्मीरागरुवासिताङ्गरुचिरे काशीपुराधीश्वरी
भिक्षां देहि कृपावलम्बनकरी मातान्नपूर्णेश्वरी ॥ २ ॥

chant, reference shardulavikridita, CER 0.031

Bṛhat Parāśara Horā Śāstra 1.3 (anuṣṭubh)

इति स्तुत्या सुसंहृष्टो मुनिस्तत्त्वविदां वरः।
अथादिदेश सच्छास्त्रं सारं यज्ज्योतिषां शुभम्॥ ३॥

chant, reference anushtubh, CER 0.0

Jaimini Sūtras 1.1.1 (prose)

उपदेशं व्याख्यास्यामः॥

read, reference anuṣṭubh, CER 0.0

One sūtra, two references

Nyāyasūtra 1.1.2 read twice. Vāgdhenu ships two prose references (gadya) and its author reads prose against the anuṣṭubh reference instead, finding that gadya drops onsets. vachaka’s prose default follows the author. Judge for yourself:

anuṣṭubh reference

gadya reference

दुःखजन्मप्रवृत्तिदोषमिथ्याज्ञानानामुत्तरोत्तरापाये तदनन्तरापायादपवर्गः

Numbers

Measured on an Apple M3 Max (36 GB), with the DiT on MPS and the vocoder on the CPU.

loading Vāgdhenu	10 to 17 s
rendering, warm	about 5.5 s per second of audio (a 9.5 s anuṣṭubh verse in 50 to 57 s)
share of render time in the DiT	77% (the rest is BigVGAN on the CPU)
loading Su-śrotā	9 s
transcribing a 10 s clip on the CPU	0.4 s
alignment resolution	40 ms (Su-śrotā’s frame)

At that speed an hour of recitation takes five to six hours on a laptop. A CUDA GPU is much faster; Vāgdhenu’s author renders on one.

The pilot rendered 27 sections (293 s of audio) from a stotra, three jyotiṣa books, the Jaimini Sūtras and the Nyāyasūtra, taking the best of up to three seeds per section:

voice	sections	mean CER	worst	exact
chant, reference for the detected metre	9	0.030	0.086	2
prose, anuṣṭubh reference	14	0.016	0.118	9
prose, gadya reference	4	0.010	0.039	3

Two cautions. Su-śrotā comes from the same lab and its training data includes a TTS-speaker slice, so it may hear Vāgdhenu more kindly than a person would; a low CER says the words are there, not that the chant is beautiful. And a section whose metre is not detected (āryā, a verse split over three lines) is read as prose, which sounds flatter than a chant. Listen before you ship.

Install

pip install vachaka            # rendering
pip install 'vachaka[check]'   # Su-śrotā: transcripts, error rate, word timings

Python 3.10 to 3.12. The first Chanter() downloads about 1.5 GB of weights from the Hugging Face Hub and fetches the sources of Vāgdhenu, BigVGAN and IndicF5 at pinned commits into ~/.cache/vachaka (set VACHAKA_CACHE to move it); none of the three is on PyPI, and the f5-tts that is differs from IndicF5. Su-śrotā is 520 MB more. Writing mp3 needs ffmpeg.

Use

Chant a verse given as its lines, and read prose:

from vachaka.core import Chanter
ch = Chanter()                        # cuda, then mps, then cpu
r = ch.chant(['वसुदेवसुतं देवं कंसचाणूरमर्दनम्', 'देवकीपरमानन्दं कृष्णं वन्दे जगद्गुरुम्'])
r.spans                               # [(0, 4230), (4980, 9550)]: one span per line, in ms
r.save('krishna.mp3')
ch.read('अथातो धर्मजिज्ञासा । चोदनालक्षणोऽर्थो धर्मः ।', ref='anuṣṭubh').save('mimamsa.mp3')

Check a render and time its words:

from vachaka.check import Ear, best_of
ear = Ear()
lines = ['वसुदेवसुतं देवं कंसचाणूरमर्दनम्', 'देवकीपरमानन्दं कृष्णं वन्दे जगद्गुरुम्']
r = best_of(lambda seed: ch.chant(lines, seed=seed), ear, ' '.join(lines))
r.cer                                 # 0.0
ear.hear(r.audio, r.sr)               # 'वसुदेवसुतं देवं कंसचाणूरमर्दनं देवकीपरमानन्दं कृष्णं वन्दे जगद्गुरुं'
ear.words(r, [l.split() for l in lines])
# [[['वसुदेवसुतं', 120, 1279], ['देवं', 1279, 1878], ['कंसचाणूरमर्दनम्', 1878, 3756]],
#  [['देवकीपरमानन्दं', 5100, 7018], ['कृष्णं', 7018, 7697], ['वन्दे', 7697, 8217], ['जगद्गुरुम्', 8217, 9056]]]

Recite whole sections from their display lines, with daṇḍas, verse numbers and accents left in; each line comes back with its span and words:

from vachaka.recite import recite
rec, info = recite(ch, ear, [dict(lines=['नित्यानन्दकरी वराभयकरी सौन्दर्यरत्नाकरी', 'निर्धूताखिलघोरपावनकरी प्रत्यक्षमाहेश्वरी ।',
                                         'प्रालेयाचलवंशपावनकरी काशीपुराधीश्वरी', 'भिक्षां देहि कृपावलम्बनकरी मातान्नपूर्णेश्वरी ॥ १ ॥'])])
info[0]['kind'], info[0]['ref'], info[0]['lines'][0]   # 'chant', 'shardulavikridita', {'s': ..., 'e': ..., 'w': [...]}

For batches there is a resumable CLI over jsonl: vachaka-recite jobs.jsonl results.jsonl, where a job is dict(id, out, sections).

How it works

recite groups display lines into pieces, closing a piece after a daṇḍa, so the two pādas of a hemistich printed on two lines render together, as Vāgdhenu expects. Each verse (closing on a double daṇḍa) is scanned by ganapati: one in a metre from Vāgdhenu’s bank is chanted against that metre’s reference clip, another vṛtta against vasantatilakā, and an āryā or anything that does not scan is read in chunks of at most 24 akṣaras. So a commentary that quotes a verse chants the verse and reads the rest. Vāgdhenu renders each piece separately and joins them with short silences; vachaka records where each piece landed. Su-śrotā then transcribes the whole render, and if the error rate is above 8% the section is rendered again with another seed, keeping the best of three. Finally the words of each piece are force-aligned (torchaudio’s CTC aligner on Su-śrotā’s posteriors) inside that piece’s span, so one bad line can’t pull the rest out of place.

Two things had to be patched to run this on a Mac. BigVGAN’s transposed convolution has more than 65,536 output channels, which MPS rejects, so the vocoder runs on the CPU. And Su-śrotā’s checkpoint names AI4Bharat’s NeMo fork; vachaka rewrites its config for upstream NeMo (the tokenizer type becomes agg, the fork’s multisoftmax keys go) and uses only the encoder and the CTC head, the Sanskrit columns 4096 to 4351.

Limits

No Vedic svaras: Vāgdhenu is laukika chant, and accents are stripped before rendering. One voice, the model author’s. The reference clip sets pace and melody but not timbre. Metres outside Vāgdhenu’s bank fall back to vasantatilakā when detected as some other vṛtta, and to prose when not detected at all. Prose sounds like careful reading, not chant.

Credits and licences

vachaka is Apache-2.0. Vāgdhenu (code Apache-2.0; weights redistributed under IndicF5’s MIT and NVIDIA BigVGAN’s licence) and Su-śrotā (fine-tuned from AI4Bharat’s IndicConformer, MIT) are by Prathosh A. P.; please cite Vāgdhenu if you use its audio. The model card asks that the voice not be used to impersonate anyone. Say that the audio is synthetic wherever you publish it.