Apple's RISED tops mean pass rate with rubrics

Training a single LLM agent jointly across diverse interactive environments has attracted increasing attention as a route to generalist agents. Existing curriculum and data-selection strategies often allocate training at the environment level or prioritize local reward-based signals, without considering relationships between current rollouts across environments. As environments are learned at different rates, all-failure and all-success rollout groups can coexist within a batch, leaving those data without group-relative reward signals.

To address these limitations, the researchers repurposed rubrics beyond their use as reward. An LLM judge tags each rollout using a predefined rubric vocabulary shared across environments. The resulting profiles guide data selection to align with the overall behavioral composition of the mixed-environment batch while limiting overlap with already-selected data. Positive rubrics provide privileged context for an on-policy self-distillation teacher, and negative rubrics guide subsequent rollout generation away from recurring failure modes.

The outcome hinges on whether the rubric vocabulary can generalize across new environments without manual redefinition, and on the LLM judge's tagging accuracy. For AI teams training agents across multiple simulated environments, this approach could offer a way to improve performance when scalar rewards provide insufficient contrast.