AIToday
Large Language ModelsTHE DECODERPublished: May 31, 2026, 19:00 JST

Study reveals leading AI search agents rely on memorized knowledge rather than genuine web research, with performance collapsing on questions requiring current information

Send on LINE
Study reveals leading AI search agents rely on memorized knowledge rather than genuine web research, with performance collapsing on questions requiring current information

3 Key Points

  1. Researchers from Harbin Institute of Technology and Xiaohongshu tested eleven AI models including GPT-5.4, Gemini 3.1 Pro, Claude Sonnet 4.6, DeepSeek-V4-Pro, and Kimi-K2.6, finding that models solved surprisingly high percentages of BrowseComp benchmark tasks without any internet access—MiniMax M2.5 reached 44.5 percent from memory alone, and Kimi K2.6 hit 62 percent on the Chinese BrowseComp-ZH variant.

  2. When search results contained no supporting documents, every model performed worse than without any tools at all; MiniMax M2.5 dropped from 44.5 to 8.0 percent, and Kimi-K2.6 fell from 25.5 to 2.3 percent, showing that search actively pulls agents away from correct answers when confirming information is unavailable.

  3. On LiveBrowseComp—a new benchmark containing 335 questions dependent on facts from the 90 days before creation—all models in closed-book testing fell below two percent accuracy, and with tools enabled scores landed about 25 to 40 points below the same models' BrowseComp results, with rankings shifting significantly: DeepSeek v3.2 rose from bottom on BrowseComp to top on LiveBrowseComp.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleTurkey's hair-transplant industry evolved from medical tourism foundation into a massive global market through innovation in surgical techniques and institutional knowledge-sharing.