Researchers release first benchmark for extracting claims from text-and-image posts, exposing AI gaps in fighting social media misinformation

Researchers published the first standardized test (benchmark) for extracting factual claims from social media posts that mix text with images—memes, screenshots, photos. Today's fact-checking AI typically handles text alone; this work measures how well modern multimodal AI systems (those that read both text and images together) can identify claims worth fact-checking in real-world messy posts from platforms like Twitter or TikTok.

Current state-of-the-art multimodal AI systems fail at understanding the rhetorical intent (sarcasm, metaphor, emotional manipulation) and context clues embedded in text-image combinations. The researchers introduced MICE, a framework designed to fix this by training AI to recognize when images and text are being used together to mislead—not just extracting what the post literally says, but what it intends to persuade readers to believe.

For fact-checkers and platform moderation teams, this means automated systems can now be measured on their ability to catch the same kinds of false claims that actually spread on social media (not just clean, formal text). This closes a blind spot: AI fact-checkers trained only on text miss misleading memes and manipulated photos, which are among the fastest-spreading misinformation formats online.