aimock open-sources API mock for coding agents; CacheBlend cuts multi-doc inference 2–4x faster

The article addresses two distinct but related pain points in modern AI development: test infrastructure for coding agents and inference efficiency for large language models. The aimock project solves a classic testing debt problem—when teams run hundreds of CI jobs daily, calling real APIs accumulates substantial costs, but using static mock responses creates a false sense of security because provider APIs evolve. aimock's insight is that one central repository can absorb the cost of daily real API validation on behalf of thousands of downstream projects; the corrected schema ships via npm, making the fix automatic once teams upgrade. This model distributes both the cost and the confidence across an ecosystem.

CacheBlend addresses a complementary efficiency problem in inference. Modern systems use prompt caching to make cached input tokens up to 90% cheaper, but the strict byte-for-byte prefix requirement breaks down in practical workflows—retrieving multiple documents for RAG, reordering context, or extending conversation history all trigger full cache misses. Alibaba Cloud's telemetry shows that 77% of cache hits come from only 10% of the KV cache blocks, meaning the other 90% sits unused because prefix matching prevents their reuse. CacheBlend reframes the problem as document-level caching: since tokens overwhelmingly attend to local context within a document, and only boundary tokens cross document lines, the system can reuse each document's cache independently and selectively recompute only those few boundary tokens. This shift from prefix-based to document-based caching recovers the unused 90%, yielding 2 to 4x speedups on multi-document queries and eliminating reordering penalties entirely.