Choosing speech models for Indian languages

Every Vaakyo call runs through three models: a transcriber turns the caller's speech into text, a language model decides what to say, and a voice turns the reply into speech. For Indian languages, the transcriber and the voice make the biggest difference to how the call feels.

The transcriber (speech to text)

Compare transcribers on your callers, not on benchmarks. What to look for:

• Language coverage. Does it support the languages your customers speak, including code-mixed Hinglish?
• Names and numbers. Does it get Indian names, localities, order numbers and amounts right?
• Short answers. Phone calls are full of "haan", "ji", "nahi", "theek hai". Some models drop them.
• Noise. Callers are in traffic, shops and kitchens.

Vaakyo lets you pick a transcriber per agent and set a fallback from another provider, so one provider's outage or capacity limit doesn't fail the call. Add keyterms for your brand, product and place names.

The voice (text to speech)

Listen for:

• Accent. An Indian voice for Indian callers, ideally one made for the language rather than a generic multilingual voice.
• Numbers and dates. "₹4,200" should sound like a person saying it, in the caller's language.
• Code-mixing. In Hinglish, English words inside Hindi sentences should sound natural, not like a different speaker.
• Latency. The time to the first sound matters more than the total generation time.

Try two or three voices with the same script before deciding, and play them to someone who speaks the language natively.

The language model

For most calls a fast model is the right choice: callers notice delay more than they notice a slightly less clever answer. Use a larger model only where the conversation genuinely needs it (complex support, long negotiations), and check the cost per minute before you switch; premium models cost more per minute.

Test on real calls

Pick ten real-sounding scenarios for your use case, including a noisy caller, a fast talker, a caller who switches language and one who gives one-word answers. Run them with each combination you're considering, read the transcripts and listen to the recordings. The best choice for Tamil payment reminders may not be the best for Hinglish clinic bookings, and that's fine: each agent can have its own models.