AIToday
Large Language ModelsAI Safety & AlignmentarXiv cs.CLPublished: Apr 30, 2026, 13:00 JST

MATH-PT benchmark dataset introduces 1,729 mathematical problems in European and Brazilian Portuguese to address linguistic bias in LLM reasoning evaluations

Send on LINE

3 Key Points

  1. Researchers released MATH-PT, a dataset of 1,729 mathematical problems written in European and Brazilian Portuguese, sourced from mathematical Olympiads, competitions, and exams from Portugal and Brazil.

  2. Frontier reasoning models (advanced LLMs that perform complex logical inference) showed strong performance on multiple choice questions compared to open weight models, but their performance decreased for questions with figures or open-ended questions.

  3. The benchmark addresses a significant gap: most mathematical reasoning evaluations are exclusively in English or translated from English, introducing linguistic bias into how LLMs are assessed on math tasks.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleRaMP framework optimizes Mixture-of-Experts inference by dynamically selecting kernel configurations based on runtime expert routing, achieving 1.22x kernel speedup and 1.30x end-to-end speedup in vLLM serving