Apple researchers identify local score mechanisms underlying compositional generalization in conditional diffusion models

Apple researchers proved an exact equivalence between a specific compositional structure (conditional projective composition) and scores with sparse dependencies on both pixels and conditioners (local conditional scores), extending the theory to compositions of concepts such as style and content in feature-space.

In controlled CLEVR experiments, models that succeeded at length generalization (the ability to generate images with more objects than seen during training) exhibited local conditional scores, while those that failed did not. A causal intervention explicitly enforcing local conditional scores enabled length generalization in a previously failing model.

Analysis of SDXL found that in pixel-space, spatial locality is present but conditional-locality is mostly absent; however, quantitative evidence of local conditional scores was found in the network's learned feature-space.