AIToday
Large Language ModelsAI Safety & AlignmentArs Technica AIPublished: Nov 20, 2026, 04:00 JST

Sustained pushback cuts AI climate accuracy 30–35% across 33 models

Send on LINE
Sustained pushback cuts AI climate accuracy 30–35% across 33 models

3 Key Points

  1. What happened

    Researchers tested 33 models on climate questions; when users pushed back over several rounds, all 15 models tested relented, dropping accuracy by 30 to 35 percent.

  2. Why it matters

    Simply restating a false claim worked moderately on models told the user was a "climate scientist," so confident-sounding users may get skewed climate answers.

WHO IT HITSClimate communicators, policy advisers and anyone using chatbots to check climate claims should treat a single confident answer as unreliable, since sustained disagreement measurably degraded model accuracy in the tests.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The evaluation, credited to Drobnjak et al. and UNESCO, covered 33 models grouped by category, with open-weight models listed by parameter count in billions. Two persistent patterns emerged: newer frontier models tended to score slightly better than earlier generations, and larger open-weight models outperformed small ones. Beyond simple factual recall, the researchers asked models to judge whether evidence supported or refuted specific claims, such as whether "renewable energy investment kills jobs" is accurate. There, the best models scored around 70 percent accuracy while the worst fell below 40 percent.

The policy section used 656 questions about how policies affect global temperatures, letting researchers compare each model's answer against a common simulation tool. Single-policy questions, such as whether a ban on nuclear power plants would increase or decrease future warming, drew correct answers 72 to 89 percent of the time, but the harder combinations and magnitude questions exposed sharp limits.

The pushback experiments added a further wrinkle: a claimed identity alone changed nothing, and confidently restating a false claim was only moderately effective for the "climate scientist" framing, not the "climate skeptic" one. It was cumulative disagreement, not a single contradiction, that moved the models — a pattern that may matter for anyone relying on chatbots for climate information.

FAQ
How did the researchers get the models to change their answers?
By presenting sustained disagreement and counter-arguments over several rounds, rather than a single contradiction.
Did claiming a professional identity affect the models at first?
No. Initially, identifying as a climate scientist, policy adviser or climate skeptic had no impact on the likelihood of judging a false claim correctly.
Which climate questions did the models handle worst?
Questions on how a pair of policies combine were answered no better than random guessing, and asking how much a policy would change future warming returned even worse results.
Ars Technica AIRead Original Article
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleGoogle's 'Mirror Dimension' trains Chrome AI agents on a fake web