UniME method proposed for brain tumor segmentation with missing MRI modalities using two-stage architecture combining single ViT encoder and modality-specific CNN encoders

Researchers propose UniME, a two-stage method that decouples representation learning from segmentation to handle incomplete multimodal MRI scans for brain tumor segmentation. Stage 1 pretrains a single ViT Uni-Encoder with masked image modeling; Stage 2 adds modality-specific CNN Multi-Encoders to extract fine-grained features.

The method fuses global representations from the Uni-Encoder with high-resolution, multi-scale features from the Multi-Encoders to produce segmentations when one or more MRI modalities are missing.

Experiments on BraTS 2023 and BraTS 2024 datasets show UniME outperforms previous methods under incomplete multi-modal scenarios. Code is made available.