Which search providers scored highest in the benchmark?

Parallel led with 75 points, followed by Exa with 74 and Firecrawl with 73. Without search access, the baseline model scored just 33 points.

How much can better search quality reduce AI agent costs?

Parallel's advanced version cut token consumption by over 40 percent compared to its Basic version, lowering total cost per task from $0.11 to $0.084.

What three test suites make up the Search Index?

DeepSearchQA (900 research questions), BrowseComp (200 hard-to-find facts requiring multi-step browsing), and AA-Omniscience (600 questions across six knowledge domains).

Large Language ModelsTHE DECODERAug 18, 2026Aug 19, 2026, 04:00 JST

Search Index benchmark ranks AI agent APIs on quality, cost, speed

Send on LINE

3 Key Points

What happened
Artificial Analysis released the Search Index, a benchmark testing seven search API providers (Parallel, Exa, Firecrawl, You.com, Tavily, Keenable, and Brave) using the same AI model (GPT-5.6 Luna) in identical agent setups. The benchmark combines three equally weighted test suites: DeepSearchQA (900 research questions), BrowseComp (200 hard-to-find facts), and AA-Omniscience (600 questions across six domains).
Why it matters
Search quality directly affects both AI agent performance and cost. Without search, the model scored just 33 points; with search, scores ranged from 65 to 75. Better search quality reduces total token use—Parallel's advanced version cut token consumption by over 40 percent compared to its Basic version, lowering total task cost from $0.11 to $0.084 despite higher per-search fees. This means businesses can optimize both accuracy and spending by choosing the right search provider.
What to watch
Parallel, Firecrawl, and Parallel (turbo) achieved the best cost-performance mix, though raw speed per query does not always improve total task time—Parallel's turbo version responded fastest per query (0.51 seconds vs. 1.03 seconds for Basic) but lower quality (67 vs. 73) forced more agent passes, keeping overall time flat. The methodology is public and other providers can apply to join.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, in your inbox every morning.

Get news like this every morning, free

Context & Analysis

Artificial Analysis built the Search Index to address a practical gap: how do you compare search APIs for AI agents when quality, cost, and speed all matter? By standardizing the model (GPT-5.6 Luna), the framework (Stirrup), and the test scenarios (900–600 questions per suite), the benchmark isolates the search provider as the only variable. This controlled approach yields a counterintuitive insight: the fastest search per query does not guarantee the fastest overall task completion. Parallel's turbo version delivers the shortest response time (0.51 seconds), but its lower quality forces the agent to run additional passes, negating the speed advantage. Meanwhile, Parallel's advanced version and Exa prioritize quality over raw speed, reducing the number of queries needed and cutting token consumption—which translates to lower total cost despite higher per-search fees. The benchmark thus reveals that for AI agents, quality is an economic lever: fewer, better-informed queries cost less overall than many cheap, low-quality ones.

FAQ

Which search providers scored highest in the benchmark?: Parallel led with 75 points, followed by Exa with 74 and Firecrawl with 73. Without search access, the baseline model scored just 33 points.
How much can better search quality reduce AI agent costs?: Parallel's advanced version cut token consumption by over 40 percent compared to its Basic version, lowering total cost per task from $0.11 to $0.084.
What three test suites make up the Search Index?: DeepSearchQA (900 research questions), BrowseComp (200 hard-to-find facts requiring multi-step browsing), and AA-Omniscience (600 questions across six knowledge domains).

THE DECODERRead Original Article

Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Get it free

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.