New research shows MoE language models are sensitive to demographic bias in routing but nearly impossible to fix without sacrificing performance.

Researchers introduced FARE, a diagnostic framework testing fairness interventions across MoE models including Mixtral, Qwen1.5, Qwen3, and DeepSeekMoE

Routing-level bias fixes proved either unachievable, statistically unreliable, or caused significant utility losses of up to 6.3% on TQA and 4.4% on CrowS-Pairs metrics

Bias mitigation at the routing level failed to transfer to actual text generation, with null results across all generation quality metrics

Expert groups contain deeply entangled bias and core knowledge, making it structurally difficult to remove stereotypes without damaging model capabilities