I-JEPA hits 79.3% vs MAE's 77.2% after 300 epochs

I-JEPA sits within a line of self-supervised methods that learn from unlabeled images by solving pretext tasks. Earlier approaches like MAE reconstruct masked pixels, while data2vec predicts representations but uses a single target. I-JEPA combines multiple large target blocks with a spatially distributed context, and uses an exponential moving average to update the target encoder. This design encourages the model to capture object shape and part arrangement rather than pixel-level detail. The paper's experiments show that this representation-prediction objective yields features that are easier to read out linearly: on ImageNet with frozen encoders, I-JEPA's ViT-H/14 reached 79.3% top-1 after 300 epochs, versus MAE's 77.2% after 1600 epochs. Efficiency gains also come from processing only visible patches in the context encoder and using a narrower predictor. However, the method has limits: it targets static images and does not model temporal or causal relationships, and its advantage may not hold for tasks requiring fine-grained anomaly detection.