Open-source AI models capture frontier traffic; mixture-of-experts shifts cost boundary

In 1989, Boris Yeltsin visited a supermarket in Houston and was amazed by the variety of ice cream available. OpenRouter, the author suggests, is now the AI equivalent: a marketplace where different models coexist and users choose based on their specific needs.

Data from OpenRouter reveals a surprising market structure: OpenAI's GPT-OSS-120b, a year-old open-source model, still commands 36% of the daily token volume of Anthropic's Opus 4.8, a frontier model that shipped weeks ago. Separately, GLM 5.2 now serves 495B tokens a day. This persistence of older or less-capable models is not a sign of market failure; rather, it reflects segmentation. Buyers vary in their needs—cost, speed, accuracy, size, origin (US versus China), architecture (dense versus sparse), accuracy specialization (coding or general), token speed, and modality (text-only or vision)—and they are choosing models that fit their constraints rather than all converging on a single best option.

Competition is actively shaping this segmentation. Last week, Anthropic shipped Opus 5, described as smaller and cheaper than Fable, explicitly to contest the ground that Moonshot's Kimi 3 targets. Poolside launched Laguna S 2.1, positioned as a US mid-market model. This is a market behaving competitively: vendors are building for specific niches.

A crucial breakthrough is the mixture-of-experts architecture. The author tested Laguna S 2.1, a 118-billion-parameter model with a sparse architecture, against Gemma 4 26b on an M5 Max machine. Although Laguna is much larger, it generates at the same speed because only 8 billion parameters activate per token—even though all 118 billion are stored in memory. This means a 118-billion-parameter model now runs at the decode cost of a 26-billion-parameter model. The practical payoff: the author's coding and email agent, which uses tool calls, saw tool-call failure rates drop from 29.4% (with Gemma 4 26b) to 20.1% (with Laguna), a 7-percentage-point improvement. Frontier-class quality is now accessible at local-deployment cost.

Segmentation, the author concludes, is a sign of a healthy market. The frontier still serves the world's hardest tasks, but it no longer has to serve all of them. The local tier has just received a much higher ceiling, and competition will push this trend forward.

The article illustrates a fundamental shift in how the AI market is organizing itself. A year ago, the narrative was that all traffic would eventually consolidate toward the single best frontier model; instead, the market is splintering into distinct segments, each optimized for different trade-offs. OpenRouter's data—showing that a year-old open-source model retains 36% of the traffic of Anthropic's newest frontier offering—is evidence that buyers are choosing based on cost, latency, and accuracy rather than pure capability. This segmentation is not accidental; it is being driven by intentional competition. Anthropic's decision to ship Opus 5 as a smaller, cheaper model to contest Moonshot's Kimi 3, and Poolside's launch of Laguna S 2.1 as a US mid-market option, show that vendors are explicitly building for niches rather than chasing the frontier-only race.

The mixture-of-experts architecture breakthrough amplifies this trend by collapsing a key cost boundary. When a 118-billion-parameter model can decode at the speed of a 26-billion-parameter model, it shifts what is possible in local deployment. The author's real-world test—where Laguna reduced tool-call failure rates by 7 percentage points over its predecessor—suggests that this capability gap is practical, not just theoretical. The implication is that the local tier (where models run on a user's own device or a small instance) is no longer confined to small models; it can now access much larger, more capable architectures, provided the developer can afford the memory footprint at inference time.

No comments yet. Be the first to share your thoughts!