New AGD-MBRL approach uses advantage estimates to guide diffusion models in reinforcement learning, improving long-term decision-making beyond short generation windows.

Researchers introduce Advantage-Guided Diffusion for Model-Based Reinforcement Learning (AGD-MBRL) to address compounding errors in autoregressive world models

Two guidance methods developed: Sigmoid Advantage Guidance (SAG) and Exponential Advantage Guidance (EAG) that steer the reverse diffusion process using agent advantage estimates

Theoretical proof demonstrates that SAG and EAG guidance enables reweighted trajectory sampling with weights that increase based on state-action advantage, ensuring policy improvement

Approach overcomes limitations of existing diffusion guides that are either policy-only or reward-based and myopic with short diffusion horizons