# Reducing voice agent latency · Vaakyo

> Where voice agent delay comes from (end-of-turn detection, the language model, tools, the voice) and practical ways to make calls feel responsive.

Source: https://vaakyo.com/blog/voice-agent-latency

[← Blog](https://vaakyo.com/blog)

# Why voice agents feel slow, and how to make them feel fast

A second of silence on a phone call feels much longer than it is. Where the delay comes from and the settings that bring it down.

The Vaakyo team 1 October 2026 2 min read

-   Performance
-   Engineering

In a conversation between people, the gap between one person finishing and the other replying is very short. When a voice agent takes noticeably longer, callers start saying "hello?", repeat themselves or talk over the reply. Latency is the most common reason a technically correct agent still feels robotic.

## Where the time goes

Each reply passes through four stages:

1.  **End of turn.** The agent has to decide the caller has finished speaking. Wait too little and it interrupts; wait too long and every reply starts late.
2.  **Transcription.** The last words are turned into text.
3.  **The language model.** It reads the conversation and starts writing a reply. What matters is the time to the first words, since the voice can start speaking before the whole reply is written.
4.  **The voice.** The first words are turned into audio and played.

Tools add a fifth stage when the agent has to look something up mid-call.

The call timeline in Vaakyo shows the latency of each stage for every turn, so you can see which one to work on.

## What to change

-   **Use a fast model.** For most calls, a fast model gives a better experience than a smarter, slower one.
-   **Keep the prompt focused.** Very long prompts and large knowledge pasted into the prompt slow the first words down. Move reference material into a knowledge base.
-   **Shorter replies.** Ask for one or two sentences per turn. Short replies start sooner and leave room for the caller.
-   **Fast tools.** A tool that takes two seconds adds two seconds of silence. Cache what you can, and have the agent say a short filler ("let me check") before slow lookups.
-   **Tune end-of-turn detection** for your callers. Callers who pause mid-sentence need a slightly longer wait; quick yes/no flows can use a shorter one.
-   **Allow interruptions** , so callers can cut in when they already know the answer, while short acknowledgements like "haan" or "ok" don't stop the agent.

## Measure on real calls

Measure the delay as the caller hears it: from the moment they stop speaking to the moment the agent's voice starts. Look at the slow turns rather than the average; one five-second gap is what the caller remembers.

[Read next Turn every call into structured data with analytics fields](https://vaakyo.com/blog/extract-structured-data-from-calls)

## Get started **today**

### Talk to an expert

Tell us about your calls. We'll help you plan the agent, the numbers and the rollout for your team.

[Contact sales](https://vaakyo.com/contact#sales)

### Start building

Build an agent and talk to it in your browser in minutes. New workspaces start with ₹250 of free credits.

[Start free](https://console.vaakyo.com/signup)

---

More for agents: https://vaakyo.com/llms.txt · Docs: https://docs.vaakyo.com · Sitemap: https://vaakyo.com/sitemap-index.xml