Amazon Bedrock adds cost-cutting compression for RAG applications

RAG (Retrieval Augmented Generation) applications face a cost challenge at scale: retrievers typically return 5–20 chunks, often totaling several thousand input tokens per query, all of which flow to expensive foundation models. Amazon Bedrock's query-aware compression pattern intercepts this flow with an intermediate step: a smaller, lower-cost model filters the retrieved chunks against the user's original query, outputting only the verbatim spans relevant to the question. The economics work because the savings on reduced input tokens to the primary model far outweigh the cost of running the smaller compression model, especially when the price ratio between small and large models is steep (Claude Haiku to Claude Sonnet). The pattern also reduces hallucination surface area by removing irrelevant context, an important secondary benefit for enterprise applications. Amazon Bedrock's open, composable architecture supports this as a post-retrieval customization step that layers on top of existing capabilities like prompt caching and intelligent prompt routing. The benchmark results—12% of baseline tokens and 67% of baseline cost—come from a specific corpus and query distribution; results will vary with different documents, chunk sizes, and question types.