Apple researchers introduce SFI-Bench, a video-based benchmark with over 1700 questions to evaluate spatial and functional reasoning in multimodal AI models.

Apple researchers created the Spatial-Functional Intelligence Benchmark (SFI-Bench), a video-based benchmark derived from diverse, egocentric indoor video scans, designed to evaluate two dimensions of reasoning: Structured Spatial Reasoning (understanding complex layouts and spatial representations) and Functional Reasoning (inferring object affordances and context-dependent utility).

SFI-Bench tasks include conditional counting, multi-hop relational reasoning, functional pairing, and knowledge-grounded troubleshooting, which directly test a model's ability to integrate perception, memory, and inference.

Experiments reveal that current multimodal LLMs (AI models that process both text and images) consistently struggle to integrate spatial memory with functional and external knowledge, identifying a critical bottleneck in AI reasoning capabilities.