Researchers eliminate learned routers in Mixture-of-Experts models with parameter-free Self-Routing mechanism that matches performance while improving expert balance

Self-Routing uses hidden state subspaces directly as expert logits, removing dedicated router parameters entirely from MoE layers

Evaluated on GPT-2-scale language modeling and ImageNet-1K classification tasks against learned router baselines

Achieves competitive performance with learned routers while achieving approximately 17% higher average expert utilization and more balanced expert activation

Simplifies MoE architecture by eliminating router projection while keeping the rest of the MoE layer unchanged