AWS cuts TTS first-audio latency 52.4% with Kokoro-82M streaming

The post frames the problem as a gap between an agent having an answer and a caller hearing it. A one-sentence response might generate in 600 milliseconds, but a full response takes five seconds, and a conventional integration waits for all of it before synthesis starts. The proposed fix is to stream each sentence to a hosted Kokoro-82M TTS model on Amazon SageMaker AI the moment the agent produces it.

The architecture rests on a custom container on a SageMaker AI real-time endpoint that exposes two routes: POST /invocations for one complete input and WS /invocations-bidirectional-stream for a connection kept open across many messages. The post stresses that every message arrives as a binary WebSocket frame, so a handler expecting text fails on every request, and it recommends keeping the session open between messages and closing only on an explicit close message.

On the return path, the post compares base64-encoded audio inside SSE events with multiplexing raw binary audio and JSON control frames over the same bidirectional session. The unified approach avoids the roughly 33 percent payload inflation and the frame-realignment buffering that SSE requires, though the post notes the agent-to-caller leg still uses base64 because the managed runtime accepts only JSON-serializable dictionaries.