GLM-5.2 Speculative Decoding: Configure and Validate MTP

Independent research — not an official Z.ai publication.Identity and provider disclosure

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial TeamPublished and editorially reviewed July 29, 2026

How we research and review Report a correction

Disclosed editorial pen name · AI-generated portrait

GLM-5.2 checkpoint with one next-token layer proposes five MTP draft tokens, which the target model verifies before acceptance length is measured

Original editorial diagram of the deployment path. It is not a Z.ai, vLLM, SGLang or Hugging Face product screen. The later terminal image is rendered from the archived container receipt.

GLM-5.2 ships a native multi-token prediction head. A serving engine can use that head to propose several future tokens, verify them with the target model, and commit the accepted prefix. When the draft work costs less than the target work it replaces, decoding may become faster.

The word may matters. Z.ai reports that its training changes raised acceptance length by up to 20% in one ablation. That does not mean every deployment gains 20% throughput. Draft depth, prompt shape, output entropy, batch size, tool traffic, GPU topology and engine commit all change the result. A bad profile can spend more compute verifying rejected tokens than it saves.

This guide turns the feature into an operational contract: inspect the checkpoint, copy the right engine-specific flags, run MTP off/on against the same workload, watch acceptance and user-visible latency, and retain a one-change rollback.

Separate native MTP from AngelSpec

This investigation began with an AI HOT item about Tencent’s AngelSpec announcement. The announcement is useful evidence that speculative decoding remains an active inference-engine topic. It is not evidence that AngelSpec supports GLM-5.2.

The verified GLM path comes from a different chain:

the Z.ai GLM-5.2 release post describes the model’s native MTP changes;
the official checkpoint config exposes the relevant architecture fields;
the vLLM GLM-5.2 recipe and SGLang GLM-5.2 recipe publish engine-specific launch profiles.

Do not combine unrelated draft frameworks merely because they share the term “speculative decoding.” AngelSpec’s DFly path, vLLM native MTP, SGLang’s EAGLE-style MTP flags and vLLM’s optional DSpark speculator have different weights, code paths and validation requirements. Until a project publishes a GLM-5.2 compatibility matrix or reproducible run, treat the relationship as unverified.

Read the checkpoint before choosing an engine

The model config is the first gate. Our July 29 probe pinned GLM-5.2 revision b4734de4… and observed:

Checkpoint field	Observed value	Why it matters
`architectures`	`GlmMoeDsaForCausalLM`	The engine needs the GLM MoE + DSA implementation, not a generic causal-LM path.
`model_type`	`glm_moe_dsa`	Model-family dispatch must recognize this exact type.
`num_nextn_predict_layers`	`1`	The checkpoint contains a native next-token prediction layer for MTP.
`index_share_for_mtp_iteration`	`true`	Draft iterations can reuse the DSA indexer’s selected positions as designed.
`max_position_embeddings`	`1,048,576`	This is native position capacity, not proof that a chosen cluster can serve 1M tokens.

One nextn layer does not mean “only one draft token.” The head is applied recursively across speculative steps. It also does not authorize an arbitrary depth: each extra proposal can add useful accepted work or wasted verification work. Engine support and measured acceptance set the practical depth.

Fail the deployment before downloading hundreds of gigabytes if any of these conditions is false:

assert config["architectures"] == ["GlmMoeDsaForCausalLM"]
assert config["model_type"] == "glm_moe_dsa"
assert config["num_nextn_predict_layers"] >= 1
assert config["index_share_for_mtp_iteration"] is True
assert config["max_position_embeddings"] == 1_048_576

Pin the checkpoint revision as well as the engine. A branch name such as main can move between tests even when your command is unchanged.

Translate acceptance length without inventing speed

Z.ai’s release post reports an ablation using the GLM-5.1 backbone and training data with seven MTP steps. The sequence was:

Publisher ablation stage	Acceptance length
Baseline	4.56
Add IndexShare + KV sharing	5.10
Add rejection sampling	5.29
Add end-to-end TV loss	5.47

The final arithmetic is reproducible:

absolute gain = 5.47 - 4.56 = 0.91
relative gain = 0.91 / 4.56 × 100 = 19.96%
rounded publisher claim = 20%

Acceptance length conventionally includes the bonus target token. vLLM’s current speculative metrics code computes it as:

1 + accepted draft tokens / draft iterations

That metric tells you how much of each proposal survives verification. It does not contain elapsed time, batch occupancy, interconnect cost or memory pressure. A 19.96% increase in acceptance length can coexist with a smaller speed gain, no gain, or a regression. It is not a measured throughput increase.

Label the 4.56-to-5.47 row as publisher evidence. It is not our result, not a GLM-5.2 end-to-end comparison, and not a promise for your code-generation traffic.

Preflight the vLLM 0.23.0 profile

The current GLM repository lists vLLM 0.23.0 or newer. The vLLM recipe makes speculative decoding opt-in and publishes a five-token native MTP profile. This is the copyable command our preflight serialized and parsed back into the same argument vector:

vllm serve zai-org/GLM-5.2-FP8 \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":5}' \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --served-model-name glm-5.2-fp8

The recipe pairs the profile with vllm/vllm-openai:v0.23.0. That image name is a publisher configuration reference, not a claim that our lightweight metadata container loaded the model or launched the vLLM server.

Use the JSON form as one shell argument. The vLLM speculative-decoding guide defines num_speculative_tokens as a positive integer and explains that native MTP can omit a separate draft model. Do not put sampling parameters such as temperature inside this object.

The profile passing our preflight means:

the checkpoint declares its MTP head;
the public recipe currently names vLLM 0.23.0;
method=mtp and five speculative tokens match that recipe;
shell quoting preserves the JSON;
the tool and reasoning parser names are present.

It does not mean the process will fit, kernels will compile or tool JSON will remain correct. The current vLLM recipe specifically advises using the latest main branch when MTP and tool calling are enabled together. Treat that as a versioned compatibility warning: pin the exact working commit in production, not an indefinitely moving branch.

Start with a shorter --max-model-len and low --max-num-seqs during the first load if your cluster has little KV-cache headroom. Raising either knob changes the memory test and requires another canary.

Configure SGLang as a different engine

Do not translate vLLM’s JSON key names mechanically into SGLang. Its current GLM-5.2 recipe exposes an EAGLE-style low-latency profile, while the official GLM repository lists SGLang 0.5.13.post1 or newer as the version floor:

sglang serve \
  --model-path zai-org/GLM-5.2-FP8 \
  --tp-size 8 \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 5 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 6 \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --served-model-name glm-5.2-fp8

The apparent five-versus-six difference is not enough to declare the recipes inconsistent. vLLM describes five speculative tokens in its current native MTP profile. SGLang expresses a five-step, top-k-one, six-draft-token low-latency route. Compare engine semantics and observed logs, not just one integer.

The SGLang recipe says the checkpoint’s IndexShare setting is effective with --speculative-eagle-topk 1, and tells operators to tune the draft length against acceptance length. Preserve those two constraints in your configuration review.

IndexShare creates a different constraint in Vime/Megatron pipeline training: each pipeline stage must begin on a full index-computing layer. Use the GLM-5.2 IndexShare pipeline-split validator before adapting the published PP8 layout; that boundary check is separate from serving-time draft-token tuning.

Benchmark one workload with MTP on and off

A useful test changes exactly one serving feature. Keep the same:

checkpoint revision and precision;
engine image or commit;
GPU type, count, clocks and topology;
tensor/expert parallel plan;
KV-cache dtype, context cap and concurrency;
prompt/output dataset, request order and random seed;
sampling parameters, chat template and parser flags;
warmup count and measurement interval.

Run the baseline without --speculative-config, stop it, clear only the documented benchmark state, then launch the same profile with MTP. Sequential runs avoid loading two copies and changing GPU contention.

The vLLM recipe offers this bounded starting workload:

vllm bench serve \
  --model zai-org/GLM-5.2-FP8 \
  --dataset-name random \
  --random-input 8000 \
  --random-output 1024 \
  --request-rate 10 \
  --num-prompts 32 \
  --ignore-eos

Use it to shake out the harness, not to forecast coding-agent traffic. The same recipe warns that random prompts can produce low acceptance and under-report MTP’s value. Add a second dataset drawn from your real, sanitized prompt distribution.

Store one row per run:

Field	Baseline	MTP candidate	Promotion rule
Successful responses	measure	measure	no new failures
Output tokens/s	measure	measure	improvement exceeds run variance
Median / p95 time per output token	measure	measure	neither regresses beyond budget
Median / p95 end-to-end latency	measure	measure	workload-specific limit passes
Mean acceptance length	1 by definition	measure	stable near the chosen depth
Draft acceptance rate	not applicable	measure	no collapse by prompt class
Tool-call parse pass rate	measure	measure	exactly equal or better
GPU memory and power	measure	measure	remains inside reserved headroom

Do at least three repeated runs after warmup and retain raw per-request data. One average hides tail regressions and prompt classes with poor acceptance.

Inspect the container preflight receipt

Sanitized Docker receipt showing seven passing GLM-5.2 source checks, checkpoint file totals, vLLM and SGLang MTP profiles, three rejected negative controls and an explicit no-GPU-test boundary

The terminal is reconstructed from the archived JSON result. It is not a staged server run. Source bodies, model weights, credentials and provider responses are absent.

The disposable container recorded:

Preflight case	Observed result
Public documents	7/7 returned HTTP 200
Pinned config	architecture, 1M position cap, one nextn layer and MTP IndexShare matched
vLLM contract	version floor, five-token JSON and shell round trip passed
SGLang contract	EAGLE `5 / 1 / 6` markers matched the published recipe
Zero draft tokens	rejected
Generic `draft_model` method	rejected for the native-MTP profile
Missing nextn layer	rejected
GPU serving and throughput	intentionally not run

Download the sanitized machine-readable receipt. It includes the checked source URLs, response hashes, revisions, file totals and scope flags. A status: pass in that file means “safe to begin a hardware canary,” not “safe to send production traffic.”

Budget model files before KV cache

The Hugging Face APIs reported 282 BF16 safetensor shards totaling 1,506.667 decimal GB and 141 FP8 shards totaling 755.632 GB. The current vLLM recipe lists planning minimums of 1,786 GB and 893 GB respectively.

FP8  = 893.000 - 755.632 = 137.368 GB (18.18% over file bytes)
BF16 = 1,786.000 - 1,506.667 = 279.333 GB (18.54% over file bytes)

Do not call that difference “free KV cache.” Checkpoint bytes and runtime VRAM are different quantities. Weight loading, quantization metadata, CUDA or ROCm graphs, DSA kernels, communication buffers, fragmentation, KV dtype, context and concurrent sequences all consume memory.

The local GLM-5.2 hardware guide covers the broader storage and quantization decision. For MTP, leave additional graph and verification headroom and start with a representative long-output request before raising concurrency.

Choose an initial speculative depth

The vLLM recipe’s five-token profile is a documented starting point, not an optimization result for your traffic. Use this decision ladder:

begin at the current recipe depth only on the engine version the recipe names;
log mean acceptance length and per-position acceptance;
split metrics by prompt class, output length and tool versus plain text;
reduce depth if later positions are usually rejected or latency rises;
increase depth only when acceptance remains near saturation and the engine documents the higher value;
rerun output-integrity and stability tests after every change.

For a conservative first canary, depth one can reduce the number of moving parts. vLLM’s generic MTP documentation suggests a small value as a safe starting point. The GLM-specific recipe publishes five because the checkpoint was designed for a longer native path. Which starting point is better depends on whether your immediate goal is compatibility isolation or recipe-level performance.

Record the reason with the value:

model_revision: b4734de4facf877f85769a911abafc5283eab3d9
engine: vllm
engine_version: 0.23.0
speculative_method: mtp
num_speculative_tokens: 5
selection_basis: official GLM-5.2 recipe checked 2026-07-29
promotion_dataset: sanitized-coding-agent-v3
rollback: remove speculative_config and restart the same image

Stop or roll back on six failure signals

Server rejects the profile before loading

Confirm the engine floor and checkpoint architecture. Validate the JSON as one argument and require a positive integer. Our zero-depth control failed before hardware work. Do not “fix” native MTP by supplying an unrelated draft model.

Model loads but the first long request runs out of memory

Lower --max-model-len, sequence concurrency or graph-capture scope before changing weight precision. Record startup free memory and the first representative request peak. A 1M config value is capacity metadata, not a promise that one chosen topology can allocate the KV cache.

Acceptance collapses below the chosen draft depth

Break the aggregate down by workload. Natural prose, repetitive code, reasoning and tool JSON can have different predictability. Lower the draft count when later positions are mostly rejected. If mean acceptance remains near one, disable MTP: the verifier is committing little extra work.

Tail latency rises despite a faster average

Inspect p95/p99 time per output token and end-to-end latency, not tokens per second alone. Batch scheduling and speculative verification may help throughput while hurting an interactive user. Promote separate profiles for batch and chat if their objectives differ.

Tool JSON or reasoning delimiters corrupt

Compare exact parser success with MTP off. vLLM issue #34449 documented a historical GLM-5 malformed-tool regression under MTP; later discussion describes it as fixed in 0.20.0. That does not prove a current GLM-5.2 bug, but it establishes a valuable regression test. Require valid JSON, exact tool name, schema-valid arguments and a complete reasoning boundary before promotion.

Workers hang under sustained traffic

Run a soak test longer than the expected failure interval and alert on an engine that restarts while its outer container remains alive. The open GLM-5.1 vLLM report #40926 describes MTP-related hangs under sustained traffic. It is not a measured GLM-5.2 incidence rate. Use it to design the canary: request timeout, engine health probe, restart counter and a ready rollback profile with MTP removed.

Decide between native MTP, DSpark and no speculation

Route	Extra model	First reason to choose it	First reason to reject it
Native MTP	none beyond GLM-5.2	Current official vLLM and SGLang recipes support the checkpoint’s built-in head.	Acceptance or latency fails your A/B gate; parser/stability regression appears.
vLLM DSpark	external `RedHatAI/GLM-5.2-speculator.dspark` in the current recipe	You have the exact supported image, hardware and workload to test the separate seven-token route.	Adds another checkpoint and compatibility surface; not equivalent to native MTP.
No speculation	none	Lowest-complexity control, rollback target, or workload with poor acceptance.	Leaves a measured native-MTP gain unused when latency or throughput is constrained.
AngelSpec/DFly	project-specific draft assets	Only after Tencent or an independent reproducer publishes GLM-5.2 compatibility and a complete recipe.	No GLM-5.2 compatibility evidence was found in this operation.

Keep “off” as a first-class production profile. A rollback that requires editing several unrelated flags is not a safe rollback.

Move from preflight to production canary

Use four gates:

[ ] Source gate
    exact model and engine revisions are pinned
    checkpoint exposes nextn and IndexShare fields

[ ] Load gate
    all ranks start and report healthy
    representative long prompts fit with reserved memory

[ ] Correctness gate
    plain text, reasoning and tool fixtures match the baseline contract
    malformed, truncated and duplicate tool actions remain zero

[ ] Performance gate
    repeated real-workload runs beat variance
    p50 and p95 latency stay inside their separate budgets
    acceptance stays healthy by prompt class
    a sustained soak completes without engine restart

Canary a small traffic slice and retain both profiles. Capture engine commit, GPU topology, driver, kernels, speculative config and dataset hash beside each result. A later image, model revision or parser change invalidates the old acceptance conclusion.

The decisive metric is accepted work per constrained resource—not the largest draft number or one synthetic tokens-per-second headline.

Audit the MTP sources and measurement boundary

Primary and authoritative references:

The complete sanitized probe, exact revisions, hashes, bridge failure, approved network exception and cleanup receipt live under docs/evidence/glm-5-2-mtp-speculative-decoding-2026-07-29/.

We did not use a hosted Z.ai or OpenRouter request because a managed API response cannot reveal whether its provider enabled MTP. We did not download weights or rent a qualifying cluster. Performance fields in this guide are therefore either labeled publisher evidence, reproducible arithmetic, or instructions for the reader’s own A/B run.

Answer common GLM-5.2 MTP questions

Does GLM-5.2 support speculative decoding?

Yes. The official config exposes one nextn prediction layer and MTP IndexShare, while current vLLM and SGLang GLM-5.2 recipes publish native speculative profiles. Engine and hardware compatibility still need a versioned test.

Is GLM-5.2 twenty percent faster with MTP?

That conclusion is not supported by the cited ablation. The reported number is a 19.96% acceptance-length increase, rounded to 20%, in a specified training experiment. Measure end-to-end speed on your own engine, hardware and workload.

Should vLLM use five or SGLang use six draft tokens?

Use the current recipe for the chosen engine as a starting point. The engines express their paths differently. Tune against observed acceptance and user-visible latency instead of trying to make the numbers look identical.

Can I verify MTP through the Z.ai or OpenRouter API?

No reliable client-side response field proves the provider’s internal serving configuration. You can benchmark a hosted route, but you cannot attribute its latency to MTP without provider evidence.

Do I need a separate draft model?

Not for native GLM-5.2 MTP. vLLM’s native profile omits a draft model. Optional routes such as DSpark do add an external speculator and must be tested as a separate deployment.

What is the safest rollback?

Remove only the speculative configuration, restart the exact same pinned engine image with the same model and parser flags, and route canary traffic back after health and correctness checks pass. Keep that baseline profile tested before enabling MTP.