Native MTP vs bolt-on EAGLE/Medusa for product ownership

Speculative decoding needs a guesser that proposes the next few tokens. You can bolt one onto a finished model, or the model’s creators can train one in from the start. Both end up as a similar line in a serving config. They lead to very different answers to a practical question: who has to maintain the guesser when the model changes?

Built in: Multi-Token Prediction (MTP)

DeepSeek-V3’s Multi-Token Prediction is a training objective first. At each position the model learns to predict additional future tokens through small modules chained one after another, so each extra prediction still follows the usual left-to-right logic.

The open V3 release ships one MTP module, about 11.5B unique parameters, sharing the main model’s embedding and output layers. DeepSeek’s report is clear about using it: you can throw the module away and run the main model alone, or keep it and use it as a built-in guesser.

Two training details are worth knowing. DeepSeek set depth D=1 and an MTP loss weight of 0.3 early in training, then 0.1. And their tests show the MTP objective still improves quality even if you discard the module at inference. The lab paid for a richer training signal and got an optional guesser as a bonus.

TensorRT-LLM reports DeepSeek-R1-FP4 at batch 1 on 8 B200 GPUs at about 2.16x speed with MTP guessing 3 tokens versus none, and about 2.33x with a relaxed acceptance rule during the model’s thinking phase.

Bolted on: EAGLE-3 and Medusa

Medusa adds small heads on top of the model’s last layer, either with the model frozen (Medusa-1) or tuned together (Medusa-2). Reported speedups on Vicuna: above 2.2x for Medusa-1 and about 2.3 to 2.8x for Medusa-2.

EAGLE-3 trains a separate draft that reads features from several layers of the main model. Reported ceiling: up to about 6.5x versus normal decoding (HumanEval, Vicuna-13B), roughly 1.4x over EAGLE-2, and about 1.38x SGLang throughput at batch 64.

Both are a second artifact tied to one specific version of the main model.

Who owns what

Native MTP lives in the same weights file. When the base model gets a new version, the guesser comes along. You can’t bolt DeepSeek-style MTP onto someone else’s finished model without their training setup.
Bolt-ons can be added by anyone who can run the model. But every time the base model changes, the EAGLE-3 or Medusa head has to be retrained, or at least retested. Skip that and you silently lose accepted guesses and burn checking compute.

In vLLM, the MTP path is simple: set the method to mtp, no separate draft model needed, and start with num_speculative_tokens set to 1. If the model has no MTP, the docs point you to EAGLE or a draft model instead.

An analogy: the built-in versus the aftermarket GPS

Some cars come with navigation built in by the manufacturer. Software updates arrive with the car’s own updates, and it just works. Others need an aftermarket GPS stuck to the dashboard. That works on any car, and you can pick a fancier unit, but every time the car changes you’re the one checking that the mount still fits and the wiring still works.

Try this

Open the config.json of a DeepSeek-V3 or R1 model on Hugging Face and look for a field called num_nextn_predict_layers. If it’s there and above zero, the model has native MTP. Then open a model like Llama and look for the same field. It won’t be there, which means any speculative guesser for it has to be bolted on. That one field tells you who owns the upkeep.

Use it at dinner

“Some AI models come with their own built-in ‘type-ahead’ trained in by the lab. Others need one bolted on, and whoever bolts it on has to redo it every time the model updates.”