SubQ claims 56x faster LLM processing with sparse attention

What happened

Subquadratic emerged from stealth last month claiming to have cracked a mathematical problem that has constrained large language models for nearly a decade. The company's new model, SubQ, uses sparse attention (a method that drastically cuts the number of computations needed) instead of the dense attention mechanism that powers today's most advanced models. An independent evaluation by third-party firm Appen found that SubQ was 56 times faster than models using FlashAttention, a previous sparse-attention technique, and matched the coding performance of top models from Google DeepMind, OpenAI, and Anthropic on standard benchmarks.

Why it matters

Most LLMs rely on a transformer architecture that requires multiplying every word's numerical encoding with every other word's encoding—a process that becomes exponentially more expensive as text grows longer. SubQ's sparse attention selects only the most relevant word relationships to process, which the company claims could dramatically lower costs and energy use without sacrificing performance. If SubQ's results hold up, it could reshape how companies build language models going forward and make AI applications far cheaper to operate.

What to watch

Subquadratic says SubQ can process up to 12 million tokens at once in its context window, compared with one million tokens for most top models today. The company has not yet made SubQ widely available for public testing, and cost claims are difficult to verify independently at this stage. According to the CEO, running Anthropic's Opus 4.6 through a standard test cost $2600, while SubQ cost eight dollars—but until the model is available to the broader market, this comparison cannot be independently confirmed.