Researchers discover that the modality gap in CLIP and similar models may actually improve robustness rather than hinder performance

Multi-modal models like CLIP consistently show a modality gap where image and text distributions remain separated in shared embedding spaces

Mathematical analysis reveals that contrastive loss minimization creates a global gap vector orthogonal to embeddings under certain conditions

The modality gap is monotonically related to robustness: closing it maintains clean accuracy but reduces adversarial robustness

Findings challenge the assumption that alignment of modalities is always beneficial and suggest the gap may be a feature rather than a bug