I built a sub-500ms latency voice agent from scrat

Nick Tikhonov spent six months building agent prototypes for one of the largest CPG companies in the world, and somewhere along the way he got frustrated with the existing voice agent options. So he did what any reasonable engineer would do—he built his own. From scratch. In about a day. For roughly a hundred bucks in API credits.

The result: ~400ms end-to-end latency from phone stop to first syllable. That's full STT → LLM → TTS in the loop, clean barge-ins, no precomputed responses, and it beats Vapi's equivalent setup by 2×. Let that sink in. A solo engineer with a GitHub repo just dunked on a funded platform.

Here's what he figured out that the big players are missing: voice is a turn-taking problem, not a transcription problem. VAD alone fails—you need semantic end-of-turn detection. The whole system reduces to one loop: speaking versus listening. The two transitions—cancel instantly on barge-in, respond instantly on end-of-turn—define the entire experience. STT → LLM → TTS must stream. Sequential pipelines are dead on arrival for natural conversation. TTFT (time to first token) dominates everything. Groq's ~80ms TTFT was the single biggest win. Geography matters more than prompts—colocate everything or lose before you start.

This is the kind of build that makes you wonder why we're still tolerating laggy voice assistants. Nick put the code on GitHub and wrote it up in a post that hit the Hacker News front page. If you're building AI or voice products and want hands-on help, he does focused consulting. The future of voice isn't waiting for big tech to figure it out—it's people like Nick tearing down the stack and rebuilding it right.