Apple researchers create first benchmark to test whether AI chatbots actually understand context in conversations

Apple's machine learning team published a new evaluation framework (a set of standardized tests) for Large Language Models (AI systems that generate human-like text). The benchmark includes four distinct task types across nine datasets, specifically designed to measure whether these AI systems can truly follow and remember conversation context—something previous tests largely ignored.