Real-time voice AI
Voxora
Voice agents that listen, think, and answer fast enough to feel human, on infrastructure you can actually operate.
- Year
- 2025
- Role
- Architecture & build
- Built with
- Python, LiveKit, FastAPI, Deepgram, Postgres, Redis, MinIO
- Source
- Repository
the latency trap
A voice agent that pauses for two seconds feels broken. Building one that interrupts naturally, recovers from cross-talk, and swaps its speech and language models mid-flight without a redeploy is not really an AI problem. It is a low-latency distributed-systems problem wearing an AI costume.
the pipeline
- 01
Split the platform into a real-time voice-engine and a control-plane API, so runtime workers stay hot while tenants reconfigure agents live.
- 02
Ran transport over LiveKit and made every provider swappable: Deepgram or AssemblyAI for speech, Groq, OpenAI or Anthropic for reasoning, Cartesia or ElevenLabs for voice, each with failover.
- 03
Scoped every call to a tenant and an agent config, so persona, tools, and knowledge change without touching a running worker.
- 04
Chased the latency budget end to end: streaming transcription, interruption-safe dialogue turns, a TTS sanitizer, and idle worker pools kept warm and sized to CPU load.
- 05
Wired the operational layer: recordings and transcripts to S3, a per-call cost model in USD and INR, provider key rotation, and a runbook for on-call.
how it runs now
A multi-tenant platform where production voice agents are built, tuned, and operated like any other service. New agents ship as config, not deploys, and the latency stays inside a budget you can actually hold.
0
AI layers, swappable
0
provider integrations
0
redeploys to reconfigure
signal chain