AIToday
arXiv cs.LGPublished: Apr 30, 2026, 13:00 JST

RaMP framework optimizes Mixture-of-Experts inference by dynamically selecting kernel configurations based on runtime expert routing, achieving 1.22x kernel speedup and 1.30x end-to-end speedup in vLLM serving

Send on LINE

3 Key Points

  1. RaMP is a routing-aware dispatch framework that selects the fastest kernel configuration from a runtime expert histogram, achieving 0.93% mean regret versus exhaustive search using a four-parameter wave cost model fitted from 10-24 minutes of one-time profiling per model.

  2. A performance-region analysis derived from hardware constants alone predicts when each optimization helps, correctly predicting all 8 tested architectures including 3 unseen ones. Paired with a CuTe DSL kernel exposing 134-268 polymorphic configurations, RaMP delivers 1.22x kernel speedup over static dispatch.

  3. In vLLM serving, RaMP achieves 1.30x end-to-end speedup over Triton, 1.41x over DeepGEMM, and 1.13x over FlashInfer CUTLASS. When applied to Alpha-MoE, it delivers 1.14x speedup with no source modification, since the model depends only on CTA grid geometry and is kernel-agnostic.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Next articleStudy challenges core assumption in neuro-symbolic AI: symbol grounding alone does not produce compositional reasoning