Stop stuffing giant prompts into expensive models; decompose and route instead

The article identifies a widespread architectural antipattern in AI products: the monolithic prompt. Teams build comprehensive system prompts packing every rule, example, and piece of company knowledge, then send that entire context to the most expensive available model on every single request. The instinct is to optimize by finding cheaper vendors, but the article argues this misdiagnoses the problem. The real issue is treating tokens as a free or near-free resource when they are actually metered and expensive.

The economic case is straightforward: a 4,000-token prompt re-read across ten conversation turns costs 40,000 input tokens just to repeat static instructions. At a few dollars per million input tokens, the cost per request seems negligible until multiplied across all users and all days—at which point the waste scales linearly with the product's own success. The latency cost is equally real but less obvious. A model cannot emit its first token until it has read the entire prompt, so a bloated prompt forces the model to process thousands of tokens before the user sees anything on screen. The article describes this as paying twice for the same mistake: once in dollars, once in seconds.

Qolca proposes five patterns that shift the mental model from "what could the model possibly need?" to "what does this specific step actually require?" Decomposing a complex request into separate steps (language detection, intent classification, context retrieval, reply generation) allows each step to carry only its necessary context. Retrieval replaces context-stuffing, pulling in the two or three relevant paragraphs instead of the entire knowledge base. Prefix caching bills static content at roughly a tenth of normal price. Cheap models handle classification and routing decisions. The expensive model is reserved for the one step that truly needs frontier-level fluency. The company's own production assistant exemplifies this: two parallel cheap classifiers, an optional conditional summary, a single expensive call for the reply, and deterministic code for routing—resulting in the same user experience at a fraction of cost and latency.