Researchers introduce Fed-CMP, a federated learning framework enabling multimodal AI models to train on privacy-protected distributed data without centralizing sensitive information.

Addresses the bottleneck of limited high-quality public data for training Multimodal Large Language Models (MLLMs) by leveraging private data in distributed silos

Proposes Fed-MA (Federated MLLM Alignment), a lightweight pre-training approach that keeps vision encoders and language models frozen while collaboratively training cross-modal projectors

Fed-CMP framework solves two key technical challenges: parameter interference during local projector aggregation and gradient oscillations in one-pass collaborative stochastic gradient descent

Represents a significant step toward federated pre-training rather than just fine-tuning of multimodal models, enabling privacy-preserving collaborative AI development