Apple researchers propose CLaRa framework to compress documents for AI systems

Retrieval-augmented generation has become a standard technique for enhancing language models with external knowledge, but the approach carries two significant limitations: the need to process long document contexts slows inference, and the retrieval step and generation step are typically optimized separately rather than as a unified system. Apple's CLaRa framework addresses both constraints by compressing documents into dense vectors before they reach the language model, thereby reducing computational overhead, while also training retrieval and generation components end-to-end with shared gradients. The key innovation is the SCP data synthesis method, which creates compressed vectors that remain semantically rich enough to support high-quality question-answering—a balance that prior approaches struggled to maintain at high compression rates. By demonstrating state-of-the-art results even at a compression rate of 16, CLaRa suggests that the unified optimization approach can recover performance while significantly shrinking the effective context length, which is relevant for practitioners deploying models on-device or in resource-constrained environments where inference speed and memory footprint are critical.