GLM-5.2 on AMD ROCm: SGLang Setup and Verification Gates

Independent research — not an official Z.ai publication.Identity and provider disclosure

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial TeamPublished August 20, 2026 · Editorially reviewed August 31, 2026

How we research and review Report a correction

Disclosed editorial pen name · AI-generated portrait

An audited sparse GLM-5.2 expert lattice passes through a verification gate into MI300X, MI325X, and MI355X capacity tiers, while a four-device MXFP4 experiment stays behind an amber caution boundary

Original editorial illustration: three increasing node capacities follow a shared verification gate. The separate amber branch represents an unverified four-device MXFP4 pilot, not a production recommendation.

Running GLM-5.2 on AMD is no longer a question of whether a ROCm launch command exists. The current SGLang matrix includes MI300X, MI325X, and MI355X, three performance profiles, pinned container tags, and an AMD-published MXFP4 checkpoint. The harder question is which combinations have actually crossed the upstream verification boundary.

This guide pins the SGLang cookbook, deploy configuration, and benchmark table at commit 7f8f030000b628ea2cb033e7457a13dd0ac80f99. It also pins the BF16, official FP8, and AMD MXFP4 checkpoint indexes. The audit recomputes exact tensor bytes, node-capacity shares, even per-rank weight floors, the verified profile count, and AMD’s one published MXFP4 accuracy ratio. It made zero model calls and zero GPU runs. SGLang and AMD measurements remain labeled as project or publisher evidence. The machine-readable audit receipt contains all 17 source hashes, revisions, formulas, results, and limits.

In this guide

Choose the AMD route by evidence
Separate configured from verified
Plan capacity from exact checkpoint bytes
Pin the ROCm image and artifact
Launch a conservative FP8 canary
Select one SGLang performance profile
Read MI355X benchmark receipts correctly
Gate the gfx950 correctness regression
Treat MXFP4 as a separate experiment
Keep MTP and context parallelism off
Run the AMD promotion sequence
Frequently asked questions
Sources and method

Choose the AMD route by evidence

The shortest defensible choice is 8×MI355X plus official FP8. It is the only AMD GLM-5.2 route whose three base SGLang profiles are currently marked verified: true. The project also records five speed cells for the same MI355X FP8 route and pins a ROCm 7.2 container after a known gfx950 correctness fix.

The other combinations are useful research candidates, but they do not inherit that label:

Hardware and checkpoint	Current SGLang state	Correct first claim
8×MI355X, official FP8	Three base profiles verified	Reproduce the pinned profile as a production canary
8×MI325X, official FP8	Three profiles configured, all unverified	Capacity exists; performance and correctness still need local proof
8×MI300X, official FP8	Three profiles configured, all unverified	Use only as a bounded canary, not a verified substitute
8×MI355X or 8×MI325X, BF16	Fits by weight capacity; all cells unverified	Research control when measured FP8 loss justifies the cost
8×MI300X, BF16	Single-node cells omitted	The weight floor leaves no practical runtime margin
4×MI355X, AMD MXFP4	Four cells configured, all unverified	Separate quantization experiment on the hardware AMD names

This is not a ranking of accelerator families. It is an evidence routing table. MI300X and MI325X may work well for a specific service, but a configuration entry without a verified flag or project row cannot establish throughput, tail latency, parser correctness, long-context behavior, or recovery.

AMD’s workload-optimization matrix also explains why “AMD FP8” is not one interchangeable target. MI300X and MI325X are CDNA 3 parts using the FNUZ FP8 encoding; MI355X is CDNA 4 and uses OCP FP8. MI355X adds MXFP4 support. Freeze the hardware architecture and precision in the deployment receipt instead of writing only “ROCm.”

Separate configured from verified

The pinned SGLang deployment configuration contains 19 AMD recipe cells in scope:

nine FP8 cells: three strategies on each of MI300X, MI325X, and MI355X;
six BF16 cells: three strategies on MI325X and MI355X;
four MI355X MXFP4 cells: three base strategies plus one MTP experiment.

Only three cells are verified, and all three are MI355X FP8. The other 16 are explicitly verified: false. The distinction matters because the configuration also contains plausible flags and container tags for those unverified cells. A command can be syntactically complete while the result remains unevaluated.

Treat upstream verification as a starting prior, not a transferable warranty. It says the SGLang project signed off on that cell under its test conditions. It does not prove your checkpoint mirror, image digest, firmware, network, context cap, request distribution, parsers, or quality suite.

The inverse is also important: verified: false does not mean broken. It means the checked source does not supply enough evidence to call it verified. The correct next step is a bounded reproduction with an explicit stop condition, not an optimistic rename.

Plan capacity from exact checkpoint bytes

The safetensors indexes establish a weight-only floor before any runtime is started:

Checkpoint	Indexed tensor bytes	GiB	Shards	Even weight-only floor in official recipe
Z.AI BF16	1,506,659,919,872	1,403.186	282	175.398 GiB/rank at TP8
Z.AI FP8	755,617,140,416	703.723	141	87.965 GiB/rank at TP8
AMD MXFP4	438,001,945,864	407.921	282	101.980 GiB/rank at TP4

FP8 reduces indexed bytes by 49.85% from BF16. AMD MXFP4 is 42.03% smaller than FP8 and 70.93% smaller than BF16. Those are exact artifact comparisons, not peak-memory reductions. MXFP4 uses TP4 in the current recipe, so its even per-rank floor is higher than an imaginary TP8 division would suggest.

AMD publishes 192 GB of HBM3 per MI300X, 256 GB per MI325X on the MI300 platform page, and 288 GB per MI355X. For an eight-GPU node, those marketed totals are 1,536, 2,048, and 2,304 decimal GB.

Eight-GPU node	Marketed HBM	BF16 tensor share	FP8 tensor share	Deployment meaning
MI300X	1,536 GB	98.09%	49.19%	BF16 leaves essentially no runtime margin; use FP8
MI325X	2,048 GB	73.57%	36.90%	Both fit by weight floor; recipes remain unverified
MI355X	2,304 GB	65.39%	32.80%	FP8 has verified cells; BF16 still lacks that label

This is why MI300X BF16 is not “close enough.” Its indexed tensors alone occupy 1,506.660 decimal GB of a marketed 1,536 GB node. The remaining 29.34 GB is for the entire eight-GPU runtime, not per GPU. A process that loads after extreme compromise would still lack a credible KV-cache and recovery budget.

Pin the ROCm image and artifact

The current SGLang config assigns three container tags:

Route	SGLang project image tag
MI355X FP8 or BF16	`lmsysorg/sglang-rocm:v0.5.13.post1-rocm720-mi35x-20260618`
MI355X MXFP4	`lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260728`
MI300X or MI325X	`lmsysorg/sglang-rocm:v0.5.13.post1-rocm700-mi30x-20260616`

A tag is not an immutable production identifier. Resolve it once, record the registry digest, and mirror or admit only that digest. Also pin the checkpoint revision and verify the safetensors index hash before shards enter the runtime. The FP8 index used here is revision ba978f7d347eaf65d22f1a86833408afdb953541 with SHA-256 e0fe7f28c1f853d4824e4d796374e3dacf1fe470988773952c79b063768134bf.

The minimum receipt should include:

{
  "accelerator": "MI355X",
  "architecture": "gfx950 / CDNA 4",
  "gpu_count": 8,
  "checkpoint": "zai-org/GLM-5.2-FP8",
  "checkpoint_revision": "ba978f7d347eaf65d22f1a86833408afdb953541",
  "safetensors_index_sha256": "e0fe7f28c1f853d4824e4d796374e3dacf1fe470988773952c79b063768134bf",
  "container_digest": "sha256:<record-the-resolved-digest>",
  "sglang_revision": "<record-the-exact-revision>",
  "rocm_version": "<record-the-observed-version>",
  "mtp": false,
  "context_parallel": false,
  "context_cap": 0,
  "peak_memory_bytes_by_rank": []
}

Do not archive credentials, model-access tokens, or full sensitive prompts in the receipt. Store public hashes and configuration separately from sanitized quality results.

Launch a conservative FP8 canary

Begin with the verified MI355X FP8 family but remove optional acceleration. Inside the pinned image, a conservative loopback canary is:

python3 -m sglang.launch_server \
  --model-path zai-org/GLM-5.2-FP8 \
  --tp 8 \
  --dsa-prefill-backend tilelang \
  --dsa-decode-backend tilelang \
  --chunked-prefill-size 131072 \
  --mem-fraction-static 0.80 \
  --watchdog-timeout 1200 \
  --reasoning-parser glm45 \
  --tool-call-parser glm47 \
  --host 127.0.0.1 \
  --port 30000

This is a GLM52.ai canary derivation, not a claim that SGLang benchmarked this exact stripped command. The TileLang DSA backends, TP8 placement, chunk size, memory fraction, watchdog, checkpoint, and MI355X image family come from the pinned configuration. Loopback binding and omission of MTP, context parallelism, DP-attention, and public exposure make failures easier to attribute.

The 1,200-second watchdog accommodates a large weight load; it does not justify an unbounded health timeout after readiness. Separate startup admission from steady-state health. During load, capture shard verification, rank allocation, kernel selection, and a clean failure reason. After readiness, require a short health response and refuse traffic if any rank is missing.

Run a small fixed corpus before throughput testing:

short factual and arithmetic prompts that catch obvious corruption;
multi-step reasoning tasks with exact expected answers;
code generation plus executable or static checks;
native tool calls using the same parser contract as production;
structured output and stop-token fixtures;
long-prefix retrieval at the declared context cap;
cancellation, maximum output, restart, and corrupted-shard failures.

The dedicated stop-token guide and MTP guide provide reusable parser and speculative-decoding controls. Keep those reader jobs separate from this hardware route.

Select one SGLang performance profile

After the plain canary passes, choose the upstream profile that matches the service objective. Do not combine its most aggressive fields before a matched test.

Profile	Chunked prefill	Memory fraction	Graph batch cap	Running-request cap
Low latency	131,072	0.80	default	default
Balanced	32,768	0.85	128	80
High throughput	default	0.85	256	256

All three use TP8, TileLang DSA prefill and decode, and a 1,200-second startup watchdog. “Low latency” is not a guarantee for every request mix. The large prefill chunk prioritizes long-prompt progress, while the balanced and throughput profiles shape graph and admission capacity differently.

Change one layer at a time:

Preserve the plain canary output and memory receipt.
Apply exactly one profile’s batch-shaping fields.
Replay the identical quality corpus and request-length distribution.
Compare time to first token, time per output token, total latency, accepted results, errors, peak memory by rank, and restart behavior.
Reject the profile if it improves raw tokens while breaching quality, health, queue, or cancellation gates.

If exact AMD capacity is unavailable, a different accelerator can rehearse the client harness but cannot validate ROCm kernels or MI-series performance.

Read MI355X benchmark receipts correctly

The pinned SGLang benchmark file contains five MI355X FP8 project rows. Every row uses a random 8,192-token input and 1,024-token output; the file says it flushed cache on every run. GLM52.ai did not rerun them.

Strategy	Concurrency	Project TTFT	Project TPOT	Project tok/s/GPU
Low latency	1	634 ms	13.56 ms	81
Low latency	16	5,411 ms	23.60 ms	621
Balanced	64	19,526 ms	46.50 ms	1,098
Balanced	256	117,866 ms	56.12 ms	1,044
High throughput	1,024	432,058 ms	106.44 ms	1,269

These rows show three important boundaries.

First, throughput and interactivity are different objectives. The highest reported per-GPU throughput comes with a 432.058-second TTFT at concurrency 1,024. That is a batch profile, not evidence for an interactive agent.

Second, more admitted requests did not monotonically raise per-GPU throughput. The balanced cell reports 1,098 tok/s/GPU at concurrency 64 and 1,044 at 256, while TTFT grows from 19.526 to 117.866 seconds. Admission beyond the useful knee can make both users and the system worse.

Third, no row exists here for MI300X, MI325X, BF16, or MXFP4. Do not copy the MI355X numbers across architectures, precisions, images, or checkpoint revisions. Define the service-level objective and reproduce the closest row with real prompt/output lengths.

Gate the gfx950 correctness regression

SGLang issue #28685 documents an earlier GLM-5.2 FP8 wrong-output failure on MI350X/MI355X gfx950. The upstream analysis traced it to an AITER block-scaled bpreshuffle GEMM path on ROCm 7.2: small row errors compounded across the 78-layer model, so short prompts could look plausible while in-context reasoning failed.

The SGLang cookbook says the problem is fixed in the pinned MI355X image and newer. It reports a post-fix GSM8K result around 0.96 with zero invalid outputs and 15/15 needle retrieval through roughly 118K tokens. The corresponding ROCm libraries pull request #8639 merged a deterministic scalar-FMA and accumulator-anchor fix at commit eaf0131b646eaa0231646aa271970692832bdd24.

Closed does not mean the regression fixture can be deleted. Add at least these gates to every image or compiler change:

Short versus long agreement: a trivial fact, a multi-step problem, and a long-prefix retrieval task must all pass. Short output alone missed the original failure mode.
Repeat determinism: run fixed greedy cases repeatedly on every rank and reject changing outputs or invalid tokens.
Precision control: compare FP8 with a separately trusted route on a small labeled corpus; do not grade only fluency.
Image provenance: require the exact digest containing the fix or a newer digest whose regression suite passed.
Compiler change canary: ROCm, AITER, CK, PyTorch, and SGLang upgrades each reopen the kernel boundary, even if the model files do not change.

MI300X and MI325X use gfx942 and were not identified as affected by this specific issue. That does not promote their unverified SGLang cells; it only keeps the gfx950 regression claim scoped correctly.

Treat MXFP4 as a separate experiment

AMD publishes amd/GLM-5.2-MXFP4 for MI350/MI355. Its card records ROCm 7.0.0, PyTorch 2.9.0, Transformers 5.8.1, AMD Quark 0.11, and both SGLang and vLLM as tested engines. The quantization is not a blanket four-bit conversion: it applies MXFP4 to MoE weights and dynamic activations while excluding attention, gates, the language-model head, dense MLP projections, and MTP layer 78.

The artifact is attractive for capacity. Its indexed tensor payload is 438.002 decimal GB, and the SGLang TP4 recipe’s even weight floor is 101.980 GiB per rank before runtime. AMD reports one flexible-extract GSM8K row: 94.09 for BF16 and 93.93 for MXFP4. Our arithmetic reproduces 99.83% score recovery, matching the rounded 99.8% publisher claim.

That is one benchmark from the checkpoint publisher. It does not establish coding, tool calling, structured output, long-context retrieval, safety, throughput, or application equivalence. More importantly, every current GLM-5.2 MI355X MXFP4 SGLang cell remains verified: false. The cookbook says the recipe was inferred from an earlier GLM-5.1 path and had not yet been benchmarked for GLM-5.2.

Runtime correctness adds a separate gate. A newer upstream incident reports that an MLA padding row can poison reserved KV slot 0 and surface as NaNs or repeated tokens. The dedicated GLM-5.2 SGLang slot-zero audit separates the merged SGLang-owned writer guard from the still-open fused AITER path and defines the required sentinel and rollback evidence. Do not treat a recipe cell, image label, or SGLang upgrade as proof that the effective writer is protected.

MoE routing precision is independent of that KV-cache invariant. The official FP8 checkpoint keeps 76 e_score_correction_bias tensors in FP32, while a captured quantized AITER path can downcast them during loading and again at the top-k call. Use the dedicated SGLang MoE correction-bias FP32 audit to inspect both dtype boundaries and expert-ID canary requirements before promoting an FP8 or MXFP4 route.

If the capacity reduction matters enough to test, create a new route and new receipt. Use the MXFP4-specific image, TP4, --trust-remote-code, and --kv-cache-dtype fp8_e4m3 only after reviewing the pinned custom code. Shadow the same requests against the FP8 control. Do not replace FP8 in place, and do not describe MXFP4 as supported on MI300X or MI325X; AMD’s card names the MI350/MI355 family.

Keep MTP and context parallelism off

The base AMD deployment cells omit MTP/EAGLE. SGLang issue #29785 remained open at this audit for a gfx950 MTP draft-kernel crash. The current config disables the standard AMD speculative options and says the DSA next-token draft path is not yet validated there.

One mtp-314 cell exists for MI355X MXFP4, but it is still marked unverified. Its presence is an experiment definition, not permission to add speculative flags to the verified FP8 route. Keep the initial command free of --speculative-* flags. After the upstream state changes, reproduce a pinned branch with acceptance length, rejected-draft work, output parity, graph capture, cancellation, memory, and restart metrics before considering promotion.

The same rule applies to DSA prefill context parallelism. The current SGLang configuration disables the ROCm CP path for MI300X, MI325X, and MI355X because it is not validated. A one-million-token architecture limit is not evidence that the AMD CP kernel, cache plan, or service budget can handle that window. Begin at the real production context cap and raise it only with memory and retrieval receipts.

Run the AMD promotion sequence

Use a single promotion ladder with explicit rollback:

Freeze the route. Record hardware architecture, GPU count and topology, firmware, ROCm stack, image digest, SGLang revision, checkpoint revision, index hash, parser flags, and network policy.
Verify offline staging. Confirm the expected shard count and tensor-index hash, prevent revision drift, and test clean shutdown before exposing a socket.
Pass plain TP8. Run the conservative FP8 command with MTP and CP off. Preserve outputs, parser events, peak memory by rank, load time, and errors.
Run the gfx950 fixture. Include short, multi-step, repeated greedy, and long-prefix cases designed to catch silent block-FP8 corruption.
Apply one performance profile. Add only the selected chunk, graph, and request-cap fields; replay the same corpus and load distribution.
Soak at the declared cap. Mix prompt lengths, maximum output, queue pressure, cancellation, health checks, and worker restart.
Rehearse rollback. Drain or reroute live work to the FP8 control without duplicate side effects; prove the old digest remains runnable.
Promote a small slice. Stop automatically on severe quality, parser, memory, availability, or recovery failures. Expand only after the observation window closes.

The final metric is accelerator cost or capacity per accepted result, not raw token rate. Include failed attempts, retries, queueing, startup, storage, networking, and human review. An unverified route becomes useful only when its measured benefit survives the same correctness and recovery contract as the control.

Frequently asked questions

Can GLM-5.2 run on a single MI300X?

Not the complete official checkpoint in the current SGLang recipes. FP8 uses TP8 across an eight-GPU MI300X node. One MI300X has 192 GB, while the FP8 index contains 703.723 GiB of tensors before runtime memory.

Is MI300X BF16 almost a single-node fit?

It is an artifact fit with no practical runtime margin, not a service fit. The BF16 tensors use 98.09% of the marketed 1,536 GB node total. SGLang omits single-node MI300X BF16 cells; use FP8 or separately validate a multi-node design.

Are MI325X FP8 recipes verified?

No. The current config provides low-latency, balanced, and high-throughput cells, but all three are verified: false. Use them as bounded reproduction templates and publish your own quality, memory, performance, and recovery receipt.

Does MI355X FP8 have published SGLang performance?

Yes, the pinned project file has five rows for 8K input and 1K output. They range from concurrency 1 to 1,024 and are reproduced in this guide. They are SGLang project measurements, not GLM52.ai results or a forecast for your workload.

Does MXFP4 mean GLM-5.2 fits on four MI355X GPUs?

The indexed weights average 101.980 GiB per rank at TP4, and SGLang defines TP4 cells. But all current GLM-5.2 MXFP4 cells are unverified, and runtime memory adds to the weight floor. Treat four-device fit as a hypothesis to measure, not a production guarantee.

Should MTP be enabled on AMD for the first launch?

No. The base AMD matrix keeps MTP off, and a gfx950 draft-kernel issue remained open. Establish a plain FP8 baseline first. Even the special MI355X MXFP4 MTP cell is marked unverified.

Is this an AMD-versus-NVIDIA benchmark?

No. The page audits AMD recipe selection and gates. For the distinct NVIDIA Blackwell quantization route, use the GLM-5.2 NVFP4 SGLang guide. Its checkpoint, backend, topology, acceleration, and benchmark contracts are different.

Sources and method

The proactive research route began after the current AI HOT batch produced no direct, non-duplicate GLM-5.2 reader task. One lead concerned Claude Code habits and had a weak GLM link; the other concerned a Qwen UI agent and overlapped the existing computer-use intent. Neither feed summary was used as article evidence.

The factual path instead pins the SGLang GLM-5.2 cookbook, its deployment configuration, its benchmark rows, three checkpoint indexes, AMD hardware documents, two SGLang issues, the ROCm fix, the official Z.AI GLM-5.2 release, and Zhipu’s research index.

The committed audit records URL, byte count, and SHA-256 for 17 source snapshots. Its calculation script derives binary and decimal checkpoint sizes, official-recipe per-rank floors, byte reductions, marketed-node HBM shares, the 19-cell verification split, and AMD’s publisher accuracy ratio. The public receipt is byte-identical to the committed result.

No checkpoint shard was downloaded, no container or server was started, no AMD GPU was accessed, and no throughput or accuracy value is represented as a GLM52.ai hardware test. The broad local hardware guide remains the place to decide between workstation experiments, accelerator clusters, and managed routes; this page owns only the AMD SGLang deployment and verification decision.