AIToday
Large Language ModelsAI WeeklyPublished: Nov 18, 2026, 13:00 JST

Pohang University: LLMs design sparse lookup nets that beat transformers

Send on LINE
Pohang University: LLMs design sparse lookup nets that beat transformers

3 Key Points

  1. What happened

    Pohang University researchers Hyunjong Ok and Jaeho Lee report that Claude Opus 5.5, GPT-6.1 Sol and GPT-6 Astra, asked to write neural-network weights directly without gradient training, all converged on sparse lookup tables. The 0.94M-parameter designs scored 54.4% on ARC versus 29.8% for 3.33M-parameter transformers from the same LLMs, with up to 17x faster GPU inference.

  2. Why it matters

    The finding suggests that models built by today's frontier LLMs may be far smaller and faster than transformer equivalents while scoring higher on ARC, which could change how teams think about model size and inference cost.

WHO IT HITSMachine-learning researchers and engineers who design or deploy compact models may need to reassess whether gradient-trained transformers are the default architecture; the result points to sparse lookup tables as a viable alternative.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The result comes from a deliberately unusual setup: instead of training a network with gradient descent, the researchers had three frontier models write the weights directly. All three independently landed on sparse lookup tables rather than transformers, which suggests the convergence is not tied to one model family.

The performance gap is striking. Despite having roughly a third of the parameters of the transformer baselines produced by the same LLMs, the lookup-table models scored 54.4% on ARC versus 29.8%. The speed difference is also large, with up to 17x faster GPU inference.

A practical consequence is traceability: because every prediction maps back to a written fact row, local edits change only about one answer in 3,530. That property may matter for teams that need to inspect or adjust model behavior without retraining.

FAQ
How much smaller are the LLM-designed models than transformers?
The sparse lookup-table models have 0.94M parameters, compared with 3.33M parameters for the transformers produced by the same LLMs.
What makes the LLM-designed models different in how they explain predictions?
Every prediction traces back to a specific written fact row, enabling local edits that change roughly one answer in 3,530.
How much faster is inference with the LLM-designed models?
The LLM-designed models offer up to 17x faster GPU inference.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleChrome adds Gemini button to sidebar for page summaries