Folkbench: judge AI relay APIs on 4 coding tests

Folkbench's starting point is that AI relays are no longer just chat front-ends: they sit between coding clients and model providers as application-layer proxies that reassemble streams and schedule context. The use cases have shifted from single-line IDE completion to terminal agents like Claude Code CLI and OpenAI Codex CLI, and to spec-driven development.

The four criteria are framed as failure modes rather than generic network metrics. Escaping can break when a relay unescapes or re-encodes chunks, cutting multi-byte UTF-8 characters and delivering code that will not parse. Tool calls can break when structured JSON with fields like path, old_str, and new_str is mis-serialized, or when assistant tool_use and user tool_result messages are not chained across turns. Long context can break when cache_control is stripped, forcing re-prefill, or when RAG trimming or model swapping shrinks the repository the model actually sees. Output can break when buffers or timeouts cut a long module short.

The guide also notes that mid-to-large codebase contexts tend to run 100k–200k tokens, which is why it treats cache passthrough and the 150k-token needle test as core checks rather than edge cases. Folkbench says P95 latency, availability, and per-token pricing for individual relays can be compared on its site without registration.