Building resumable, multi-device AI chatbot streams with Server-Sent Events requires storing every token in the database, creating overhead for successful requests that never need the data.

Advanced chatbot features—resumable streams (refresh mid-response and recover in-progress tokens), cancellations (stop the LLM mid-response), and multi-device support (pick up in-flight responses on a second device)—are possible with Server-Sent Events (SSE, a protocol for pushing real-time updates from server to client), but each requires careful architectural decisions.

To enable resumable streams using SSE's Last-Event-ID mechanism, every token must be written to a database or token cache because stateless server replicas can route a reconnection request to a different replica than the one that started the stream. This creates database overhead: each token event from LLM providers contains substantial metadata (e.g., a single Anthropic API event contains 125 characters for just 5 characters of text delta) for relatively little text data.

For successful requests where the HTTP SSE stream stays intact and delivers all tokens, the per-token data written to the database is never needed—the full response becomes available once streaming finishes and the complete response is stored in the database.