Even GPT-4o Fails at Understanding Abstract Concepts, But a New Attention-Based Method Boosts Performance by Over 4%

Large language models like GPT-4o struggle with abstract meaning comprehension across zero-shot, one-shot, and few-shot settings on the SemEval-2021 Task 4 (ReCAM) benchmark

Fine-tuned models such as BERT and RoBERTa outperform larger LLMs at interpreting abstract concepts presented in cloze-style passages with multiple choice options

A new bidirectional attention classifier inspired by human cognitive strategies improves accuracy by 4.06% on Task 1 and 3.41% on Task 2 by dynamically focusing on relevant passages and options