6 Months of Running a Production Voice AI — What Changed, What Broke, What We'd Rebuild

Short summary

After 6 months running Loquent (a voice AI for healthcare scheduling), the authors learned that prompt architecture must scale from monolithic to modular, vendor model updates require version pinning and testing, and real-world UX problems are solved through monitoring and product iteration. Concrete lessons: modular prompts cut latency by 200ms, pinned vendor versions eliminated regressions, and async flows improved satisfaction by 17 points.

• •Prompt architecture must evolve from monolithic (4k tokens) to modular with data-injected clinic config (1.2k + 300-800 tokens); reduces latency and maintenance burden
• •Vendor model updates (Deepgram, ElevenLabs) cause unpredictable regressions; pin versions in production and test against saved call corpus before upgrades
• •Real-world UX problems require iteration: hold-state confusion (6% of calls), TTS latency spikes, async insurance checks—all solved through monitoring, streaming, and caching

Generated with AI, which can make mistakes.