A SageMaker endpoint can remain InService even while an application rejects every speech request. AWS's October 22 walkthrough of low-latency speech using Kokoro on SageMaker AI identifies a concrete cause: a mismatch in WebSocket frame handling. This source was separately checked at 1:49:16 a.m. New York time on October 23.

The example hosts Kokoro-82M in a custom container on an ml.g5.xlarge real-time endpoint. SageMaker supplies client messages as binary WebSocket frames. A Starlette handler that assumes text and calls receive_text() raises KeyError('text') on every request. A generic exception handler can still return a well-formed error frame containing only text, while the endpoint continues reporting InService. Readiness does not establish agreement on message semantics.

The container offers normal invocation and a bidirectional WebSocket route. The latter keeps the connection open across several sentences. As an agent completes each sentence, it can send that sentence to synthesis while generating later text. This is earlier synthesis of the first sentence, not concurrent synthesis of all sentences; reopening the connection after each one would restore connection overhead.

AWS measured a four-sentence test five times. Mean time to first audio byte fell from 329.3 milliseconds to 156.7 milliseconds, a 52.4% relative improvement isolated to the TTS transport. In its fuller agent test, AWS reported the bidirectional mode 2.4 times faster than batch locally and 3.5 times faster in the deployed AgentCore runtime. All figures belong to AWS's example, not a general comparison for other models and deployments.

An existing Server-Sent Events pipeline can reuse base64 audio, at the cost of roughly one-third payload inflation and buffering across transport chunks. The alternative on the model-facing leg multiplexes raw binary audio with JSON control frames. Its extra complexity should be evaluated against the actual frontend boundary.

The sample container pins an older PyTorch version affected by advisory PYSEC-2026-2286; AWS says it is fixed in PyTorch 2.10.0 and advises upgrading before use beyond a proof of value. A production acceptance test should exercise the real binary/text contract, measure first audio separately from total completion and scan dependencies. The frame-handling failure would survive a readiness probe.