Gemma 4 gets 1.91–2.19x faster with Multi-Token Prediction, Sah finds

The study by Suwesh Prasad Sah, published Sept 28 and cited here as a preprint technical report, does not address a new Gemma 4 release. It isolates one variable — whether an auxiliary drafter model that proposes the next tokens is used — while holding the target model and the hardware fixed.

Gemma 4's drafter is not a small standalone language model placed alongside the target; it draws on the target's activations and KV cache. When both proposed candidates are accepted, the target adds one more token, advancing up to three tokens per verification round. That is why the mechanism can pay off even though the drafter itself adds work.

The study's measurements cover 360 requests in total, and output validation succeeded for 172 of 180 MTP trials and 177 of 180 standard trials. The remaining 11 cases were kept in the speed totals rather than discarded, and when only correctly completed outputs were counted, the roughly twofold reduction on reasoning problems still held. The reported figures come from the preprint and were arithmetically rechecked by the article's author, who did not rerun the inference.