GLM-5.2 Expert Parallelism: EPLB Sizing and Rollout

Evan Brooks

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial Team · Published August 22, 2026 · Editorially reviewed August 24, 2026

How we research and review · Report a correction

Disclosed editorial pen name · AI-generated portrait

The deployment decision

Start with expert parallelism and zero redundant experts. Choose an EP size that fits the pinned checkpoint and preserves enough memory for KV cache, graphs, communication buffers, and reserve. Replay the real request mix, then enable EPLB only if expert skew is persistent and its measured throughput or tail-latency benefit survives quality, memory, and recovery gates. For vLLM, treat num_redundant_experts as one global count: 256 + r physical slots must divide evenly by the EP rank count. Do not copy a generic r=32 setting without that divisibility and memory check.

Original editorial illustration: sparse routes expose an overloaded rank on the left; the right side represents a candidate remap that still has to pass measured latency, throughput, memory, quality, and recovery gates.

GLM-5.2 has 256 routed experts but activates only eight for each token. That sparsity reduces compute; it does not let a serving cluster discard the other 248 experts. Every request can route differently, so the complete expert library must remain available across the expert-parallel group. Real prompts can also concentrate tokens on a few experts, leaving some ranks busy while others wait at collective operations.

Expert parallelism (EP) distributes that library. Expert Parallel Load Balancing (EPLB) observes routing and can remap or replicate popular experts. They solve related but different problems. EP is a placement decision required to serve a model of this scale efficiently. EPLB is an adaptive optimization that consumes memory, moves weights, and can be wrong for a workload that changes faster than its observation window.

This guide pins the public GLM-5.2 config and tensor index, current vLLM and SGLang source, and DeepSeek’s reference EPLB repository. Its machine-readable receipt reproduces expert-matrix bytes, EP8–EP256 layouts, and redundant-slot divisibility. The audit made zero model calls, zero GPU runs, and zero NPU runs. It establishes structural capacity and safe decision gates—not a throughput claim.

In this guide

1. Start with EP, not EPLB
2. Read GLM-5.2’s expert geometry
3. Separate TP, DP, EP, and EPLB
4. Size the base EP layout
5. Translate redundant experts into memory
6. Launch a vLLM baseline
7. Map the decision to SGLang
8. Measure before rebalancing
9. Promote EPLB with rollback gates
10. Respect platform and topology boundaries
11. Rent a valid control cluster
12. Troubleshoot common failures
13. Frequently asked questions
14. Sources and method

Start with EP, not EPLB

The safest sequence is static before adaptive:

1. Pin the model revision, runtime version, image digest, kernel libraries, precision, topology, and launch arguments.
2. Select TP and DP for attention, then derive the resulting expert-parallel group. Confirm every rank sees the intended device and fabric.
3. Run EP with no redundant experts and record per-expert tokens, per-rank work, collective time, throughput, TTFT, TPOT, HBM peak, and quality.
4. Prove that imbalance persists across representative windows rather than one burst, tenant, prompt family, or warm-up phase.
5. Introduce a divisible redundant-slot count in a canary. Change no other performance variable in that comparison.
6. Promote only if the benefit exceeds run-to-run noise and every correctness, memory, latency, and rollback gate remains green.

This order matters because load imbalance is not the only reason an EP service can underperform. A weak all-to-all backend, a wrong NIC, uneven DP queues, small batches, poor grouped-GEMM shapes, graph breaks, CPU launch gaps, or memory pressure can produce similar symptoms. EPLB does not repair those causes. Adding dynamic remapping before a static baseline merely changes more than one variable.

The null result is useful. If EP ranks are already balanced for the production mix, leave EPLB off. Zero redundant experts preserves memory for KV cache and concurrency and removes weight-transfer activity from the failure surface. “Available” is not a reason to enable an adaptive system.

Read GLM-5.2’s expert geometry

At pinned Hugging Face revision b4734de4facf877f85769a911abafc5283eab3d9, the configuration declares a glm_moe_dsa model with these placement-relevant facts:

Field	Pinned value	Deployment consequence
Hidden layers	78	Base transformer layers are numbered 0–77
First dense layers	3	Base MoE begins at layer 3, leaving 75 base MoE layers
Routed experts	256	Static EP slots must cover all 256 logical experts
Experts selected per token	8	Routing selects 3.125% of logical experts per token
Shared experts	1	Shared-expert memory and compute are outside the routed-slot math
Hidden width	6,144	One dimension of every routed expert projection
MoE intermediate width	2,048	Other dimension of every routed expert projection
Expert projections	gate, up, down	Three matrices per logical expert per MoE layer
Next-token-prediction layers	1	The checkpoint contains an additional MTP expert-bearing block
Expert groups / selected groups	1 / 1	Do not import multi-group locality assumptions without runtime evidence

The pinned tensor index contains 59,585 tensor names. Exactly 58,368 are routed-expert matrices: 19,456 gate projections, 19,456 up projections, and 19,456 down projections. The identity is 76 × 256 × 3 = 58,368.

Why 76 when the base model has 75 MoE layers? Layers 3–77 are the 75 base expert-bearing transformer layers. The index also contains layer 78, whose MTP-specific eh_proj, enorm, and hnorm tensors sit beside a complete 256-expert MoE block. The config declares one num_nextn_predict_layers. That extra block is a checkpoint fact, but a runtime may load or manage it as a separate speculative model state. Capacity planning therefore needs two clearly labeled bounds rather than an unexplained off-by-one choice.

For one logical expert in one expert-bearing layer:

3 projections × 6,144 × 2,048 = 37,748,736 parameters
37,748,736 × 2 BF16 bytes       = 75,497,472 bytes

Across the 75 base MoE layers, one logical expert family is 2,831,155,200 parameters or 5.273438 GiB in BF16. Across all 76 expert-bearing checkpoint blocks, it is 2,868,903,936 parameters or 5.34375 GiB. Those two values become the lower and upper structural estimate for one additional slot per rank. Only inspection of the loaded runtime graph can decide which applies.

The full base routed library is exactly 1,350 GiB in BF16 expert matrices; the checkpoint-wide routed library is 1,368 GiB, 97.492% of the published tensor bytes. This is why “only eight experts are active” does not translate into a small memory footprint. Eight active families across the base layers represent 42.1875 GiB of routed matrix bytes conceptually, but which eight are active changes by token, and that number is not a measurement of HBM traffic or process allocation.

Separate TP, DP, EP, and EPLB

The terms are easy to collapse into “more GPUs,” but they change different parts of the model:

Mechanism	What it partitions or replicates	Primary decision
Tensor parallelism (TP)	Dense and attention tensor operations within a TP group	Kernel shape, attention memory, and intra-group communication
Data parallelism (DP)	Attention replicas or shards and independent request scheduling, depending on TP	Concurrency and queue distribution
Expert parallelism (EP)	Routed MoE expert weights and routed tokens across the EP group	Expert locality, all-to-all traffic, and expert bytes per rank
EPLB	Logical-to-physical expert mapping, optionally with redundant physical slots	Whether measured skew is worth extra weights and remapping

Pinned vLLM EP documentation defines the normal EP size as TP_SIZE × DP_SIZE. With TP2 and DP4, expert layers span eight ranks; attention uses TP2 inside each of four DP groups. The current pinned validation source also includes prefill-context-parallel ranks in its EPLB group when PCP is enabled. Record the runtime’s reported groups instead of reconstructing a complex topology from labels alone.

DP adds another source of apparent imbalance. Each DP rank schedules its own requests, so different sequence lengths and arrival patterns can give ranks different amounts of work even if expert routing is uniform. Before blaming experts, compare request count, active sequences, prompt tokens, generated tokens, cache hits, and queue time per DP rank. Expert activation statistics should be joined to scheduler and collective telemetry by the same interval.

When vLLM combines TP, DP, and EP, it can also derive a sequence-parallel MoE path whose low-load correctness is a separate promotion gate. Use the GLM-5.2 vLLM sequence-parallel MoE audit to map the exact backend and PP boundary, reproduce the one-token state-guard collision, and grade output while per-DP-rank traffic drains. Balanced expert load cannot prove that carried hidden and residual state is correct.

Size the base EP layout

With no redundancy, 256 logical experts divide cleanly by EP8 through EP256. The following table assumes an even static distribution. The BF16 figures include routed expert matrices only; they are not full-rank HBM requirements.

EP ranks	Logical slots/rank	Base 75-layer routed BF16/rank	Checkpoint-wide 76-block routed BF16/rank
8	32	168.75 GiB	171 GiB
16	16	84.375 GiB	85.5 GiB
32	8	42.1875 GiB	42.75 GiB
64	4	21.09375 GiB	21.375 GiB
128	2	10.546875 GiB	10.6875 GiB
256	1	5.273438 GiB	5.34375 GiB

This table answers only one question: how much BF16 routed-expert matrix payload an even EP layout assigns per rank. A launch must add attention, embeddings, dense layers, shared experts, normalization, DSA indexers, the MTP path, quantization metadata, KV cache, graphs, grouped-GEMM workspaces, dispatch/combine buffers, allocator fragmentation, health reserve, and any replica or disaggregation cost.

It also does not recommend EP256. Smaller expert payload per rank comes with a larger collective domain and potentially more cross-node routing. One expert per rank is a poor outcome if every token crosses a slow fabric or the rank has too little local work for efficient grouped GEMMs. Choose the smallest EP group that fits with reserve and then benchmark the actual all-to-all topology.

For FP8, NVFP4, MXFP4, W8, or W4 checkpoints, recalculate from the exact staged tensor layout. Do not divide the BF16 table by two or four. Scales, packing, unquantized layers, shared experts, padding, fused layouts, and runtime duplication break that shortcut. The dedicated NVFP4 SGLang guide and AMD ROCm guide own artifact-specific fit; this page owns expert-slot semantics.

Translate redundant experts into memory

Pinned vLLM documentation contains a potentially confusing label: its table calls num_redundant_experts “additional global experts per EP rank.” The formula beneath it and the pinned implementation resolve the ambiguity.

The EPLB state source starts one global physical-to-logical list with the 256 original experts, then appends r redundant mappings. The default policy asserts that physical experts divide evenly across EP ranks. Therefore:

physical slots       = 256 + r
slots per EP rank    = (256 + r) / EP_SIZE
valid layout         = (256 + r) mod EP_SIZE = 0
added BF16/rank      = (r / EP_SIZE) × 5.273438 GiB  # 75 base MoE layers
checkpoint upper     = (r / EP_SIZE) × 5.34375 GiB  # if MTP block participates

To add exactly one slot per rank, choose r = EP_SIZE. That adds the same 5.273438-GiB base expert payload per rank at every EP size, but global overhead grows with the group: 3.125% at EP8, 6.25% at EP16, 12.5% at EP32, and 25% at EP64. At EP128 it is 50%; at EP256 it doubles routed-expert payload. “One more per rank” is not a constant cluster-wide cost.

The vLLM page gives r=32 as a generic large-scale suggestion. For GLM-5.2, that creates 288 physical slots and 12.5% routed-expert overhead:

EP ranks	Does 288 divide?	Physical slots/rank	Added slots/rank	Base BF16 added/rank
8	yes	36	4	21.09375 GiB
16	yes	18	2	10.546875 GiB
32	yes	9	1	5.273438 GiB
64	no	—	—	—
128	no	—	—	—
256	no	—	—	—

That suggestion is neither a default nor GLM-5.2 benchmark evidence. A valid integer layout can still be a bad deployment. The redundant copies must target experts that are predictably hot, the added HBM must not evict useful KV cache, and the remap must not cost more than the imbalance it removes.

Launch a vLLM baseline

Use the pinned runtime and hardware guide for the full GLM-5.2 launch command. The following is a topology skeleton, not a universal model-fit recipe:

Static EP control—fill in the pinned checkpoint and tested backend

vllm serve <PINNED_GLM_5_2_CHECKPOINT> \
  --tensor-parallel-size 1 \
  --data-parallel-size 8 \
  --enable-expert-parallel \
  --all2all-backend allgather_reducescatter

Here EP size is eight. vLLM’s documented default all-to-all path is the portable control; DeepEP high-throughput and low-latency modes target different prefill/decode regimes and add library and fabric prerequisites. Use one backend for the baseline, record its exact dependencies, and do not change it in the EPLB comparison.

Before traffic, verify:

• detected world size, TP, DP, PCP if present, local ranks, node count, and GPU identities match the planned graph;
• (256 + r) / EP_SIZE is an integer, with r=0 for the control;
• every rank loads the expected number of logical expert slots and the same checkpoint revision;
• NIC selection, RDMA/NVLink domain, peer reachability, locked-memory limits, collective initialization, and all-to-all backend are visible in the receipt;
• warmed peak HBM leaves the approved KV, graph, allocator, and failure reserve;
• parser flags, chat template, tokenizer, reasoning mode, tool calls, streaming, cancellation, and stop behavior pass before performance measurement.

Collect at least one full demand cycle or a replay that preserves prompt lengths, output lengths, concurrency, tenants, cache reuse, reasoning mode, tool usage, and arrival bursts. Uniform-random routing simulation is useful for a communication control but deliberately removes the skew EPLB is meant to address; it cannot decide whether production needs balancing.

If the static control shows persistent expert imbalance, create one canary with a precomputed valid r. For example, EP8 plus one redundant slot per rank uses global r=8:

EPLB canary additions for an existing EP8 command

--enable-eplb \
--eplb-config '{"window_size":1000,"step_interval":3000,"num_redundant_experts":8,"log_balancedness":true,"use_async":true}'

The pinned defaults are window 1,000, interval 3,000, r=0, balancedness logging off, and asynchronous operation on. Repeating those numbers does not make them optimal. Preserve logging only long enough to measure its overhead; choose a window that spans representative routing, and prevent rapid remaps when workload phase changes are shorter than weight-transfer recovery.

Map the decision to SGLang

The pinned SGLang GLM-5.2 cookbook provides model-specific evidence: its balanced and high-throughput strategies use DP-Attention with DeepEP. The pinned general EP page separates token dispatch from expert compute and exposes the corresponding choices:

SGLang control	Decision it represents	Gate before use
--tp and --ep	Attention tensor group and expert group	Exact checkpoint fit and rank graph
--moe-a2a-backend deepep	DeepEP dispatch/combine	Supported topology, dependencies, and measured fabric
--moe-runner-backend deep_gemm	Grouped expert GEMM implementation	Precision, architecture, and kernel compatibility
--deepep-mode normal	Prefill/high-throughput-oriented dispatch	Production prefill mix and throughput control
--deepep-mode low_latency	Decode/low-latency-oriented dispatch	Decode graph support and TPOT control
--deepep-mode auto	Runtime switching between modes	Stable transitions under the mixed workload
--enable-eplb	Activation-statistics-based placement/replication	Persistent imbalance plus rollback-safe memory

SGLang notes an important backend constraint at the pinned revision: DeepEP, Mooncake, NIXL-EP, ascend_fuseep, PPLX, and MORI require ep_size = tp_size; the none dispatcher supports hybrid EP smaller than TP. PPLX adds a DP-Attention requirement. Treat this as a revision-specific contract. A CLI that accepts flags is not proof that the chosen dispatcher, runner, precision, and topology form a supported GLM-5.2 cell.

Its EPLB documentation recommends more batch volume to stabilize activation statistics and gives periodic rebalancing as an example. It does not provide a universal GLM-5.2 threshold. DeepSeek’s reference EPLB algorithm can replicate popular experts and choose hierarchical or global balancing, but GLM-5.2 declares only one expert group. Do not assume a multi-group model’s hierarchical locality benefit carries over. Inspect actual placement and cross-node token traffic.

Do not compare vLLM and SGLang while changing checkpoint, quantization, GPU, fabric, context, concurrency, batch policy, speculative decoding, or parser behavior. First establish a valid control within each runtime. Cross-runtime comparison comes only after each one passes its own correctness receipt.

Measure before rebalancing

Balancedness alone is a diagnostic, not the objective. vLLM defines it as mean expert token load divided by maximum expert token load. A value near one means token counts are even; it does not show that all experts cost the same, ranks receive equal communication, or user latency improves.

Join these measurements over identical windows:

Signal	What it can reveal	What it cannot prove alone
Per-logical-expert tokens	Hot experts and stability of routing skew	Rank time after replication and placement
Per-rank routed tokens	Whether placement concentrates work	Communication or kernel efficiency
Balancedness	Simple normalized skew trend	Throughput or tail-latency gain
Dispatch/combine duration	All-to-all and synchronization burden	Whether the bottleneck is the fabric, queue, or compute
Expert GEMM time	Per-rank expert-compute imbalance	End-to-end service benefit
DP queue and active sequences	Scheduler-side imbalance	Expert routing distribution
TTFT and TPOT distributions	User-visible prefill and decode outcomes	Root cause without correlated traces
Accepted tokens/second	Useful service throughput	Quality, retry, or protocol correctness
Peak and steady HBM	Headroom consumed by copies and workspaces	Recovery after a rank or node event

Segment by workload rather than averaging everything together. Interactive short prompts, long-context prefill, tool-heavy agents, batch generation, and speculative decode can route and communicate differently. Preserve tenant weighting and cache state. A remap that helps a synthetic steady batch may hurt bursty interactive traffic.

Use repeated A/B windows and report dispersion. Promote only when the confidence interval for the chosen primary metric excludes zero by a margin that matters operationally. Do not cherry-pick the best window. If EPLB helps mean throughput but violates p95 TTFT or TPOT, it failed the stated service contract.

Quality and protocol checks run on both layouts. Rebalancing should be mathematically output-preserving, but a new kernel path, graph capture, quantized transfer, stale mapping, or version incompatibility can violate that expectation. Compare deterministic fixtures where possible, sampled quality distributions where necessary, and complete reasoning/tool/stream framing.

Promote EPLB with rollback gates

Write the acceptance contract before enabling the canary. A useful receipt contains:

Minimum EPLB promotion receipt

{
  "model_revision": "b4734de4facf877f85769a911abafc5283eab3d9",
  "runtime_and_image_digest": "<record>",
  "checkpoint_and_precision": "<record>",
  "tp_dp_pcp_ep": "<record>",
  "all_to_all_backend": "<record>",
  "logical_experts": 256,
  "global_redundant_experts": 8,
  "physical_experts": 264,
  "slots_per_rank": 33,
  "observation_window": "<record>",
  "baseline_metrics": "<sanitized-artifact>",
  "canary_metrics": "<sanitized-artifact>",
  "quality_and_protocol_result": "pass|fail",
  "peak_memory_and_reserve": "<record>",
  "rollback_test": "pass|fail",
  "credentials_archived": false
}

Then apply these gates in order:

1. Configuration gate. Exact revisions, physical slot arithmetic, rank graph, backend, NICs, and kernel libraries match the approved receipt.
2. Load gate. The same production-shaped replay demonstrates persistent expert or rank skew across repeated windows.
3. Memory gate. Peak HBM after warm-up, graphs, KV growth, weight movement, and recovery retains the explicit reserve. No rank relies on average memory.
4. Correctness gate. Deterministic fixtures, reasoning separation, tool schemas, streaming deltas, cancellation, and finish reasons match the control. Sampled quality stays inside the approved confidence boundary.
5. Performance gate. The primary metric improves beyond noise; p95/p99 TTFT and TPOT remain inside their separate budgets; error and retry rates do not rise.
6. Stability gate. Several rebalance cycles complete without stale maps, collective timeout, graph recapture storms, request loss, or memory creep.
7. Recovery gate. A controlled restart and rollback to static EP work from the pinned artifacts. In-flight behavior and idempotency match the service contract.
8. Canary gate. Only a small traffic slice receives EPLB. Expansion waits for the declared observation window rather than a few favorable minutes.

Stop automatically on a mapping mismatch, collective error, unexpected weight transfer, HBM reserve breach, correctness failure, or tail-latency breach. Keep the static EP deployment ready as the rollback target. Never repair a failed canary by simultaneously changing r, backend, batch size, and runtime.

Reopen the evidence after any checkpoint, tokenizer, quantization, runtime, driver, communication library, kernel, topology, context policy, MTP setting, or major request-mix change. Historical balance is not inherited by a new deployment graph.

Respect platform and topology boundaries

Pinned vLLM core validation source allows EPLB only on CUDA-like CUDA or ROCm devices, requires expert parallelism, and requires more than one rank across the relevant parallel dimensions. This page therefore does not transfer its vLLM EPLB launch contract to Ascend.

SGLang documents an ascend_fuseep dispatcher and discusses EPLB in its broader EP page, while vLLM Ascend has its own plugin, releases, flags, and hardware behavior. Use the dedicated GLM-5.2 vLLM Ascend guide for Atlas A2/A3 deployment evidence. A platform-specific plugin can add capabilities that upstream core rejects; that must be proven from that plugin’s pinned source and tested hardware, not inferred from similarly named flags.

Network topology matters even within CUDA or ROCm. Record which ranks share NVLink or equivalent local fabric, which cross InfiniBand or RoCE, and whether the all-to-all backend uses the intended HCA. Compare per-link bytes and collective time. An algorithm can produce numerically balanced expert loads while increasing expensive cross-node traffic.

Avoid broad privilege changes copied from troubleshooting examples. Diagnose locked memory, device access, RDMA registration, NVSHMEM, and container limits from the exact error and platform contract. Any host-network, device, IPC-lock, or privileged-container exception should be minimized and reviewed separately; it is not an EPLB tuning knob.

Rent a valid control cluster

After sizing the EP group and declaring the measurement contract, a rented GPU cluster can provide an isolated control. Its value depends on receiving the exact accelerator count, interconnect, node shape, storage, image support, and network access required by the experiment—not merely the same GPU model name.

Build an isolated EP/EPLB control

Use a rental only after calculating the exact rank graph and memory reserve. Pin the image and checkpoint; record every device, NIC, and link; replay sanitized production-shaped traffic; and destroy only the task-owned resources after exporting the receipt. Do not compare a single-node rental with a multi-node production layout or transfer its throughput, balancedness, and tail-latency result across fabrics.

Check current multi-GPU availability on RunPod →

Affiliate disclosure: This is an affiliate link and a referral link. GLM52.ai may receive a commission or referral credit after qualifying usage at no added charge from us. A rented GPU does not guarantee multi-node EP fabric, RDMA or NVLink topology, vLLM/SGLang compatibility, checkpoint capacity, or a reproduced EPLB result. Verify the live SKU, rank count, interconnect, node isolation, storage, egress, price, and runtime support before use.

Budget by accepted output, not accelerator-hour headline alone. Include weight download and storage, warm-up, idle reserve, failed canaries, replay generation, network charges, retries, and the static rollback capacity. A redundant expert is economical only when its measured service benefit exceeds both added HBM opportunity cost and operational complexity.

Troubleshoot common failures

The runtime rejects the redundant-expert count

Calculate (256 + r) mod EP_SIZE. For EP64, r=32 creates 288 physical experts, and 288 is not divisible by 64. Choose a global r that makes the total divisible; do not reinterpret r as “per rank.” Also confirm EPLB and EP are both enabled and that the pinned runtime supports the platform.

HBM rises more than the table predicts

The table counts routed expert matrices only. Check whether the runtime also replicates the MTP expert-bearing block, uses padded or fused layouts, retains old and new maps during asynchronous movement, captures new graphs, expands workspaces, or reduces KV-cache capacity. Quantized copies include scales and packing. Measure per rank; the maximum, not the average, controls safety.

Balancedness improves but throughput does not

The bottleneck may be all-to-all latency, dispatch layout, grouped GEMM, attention, DP queue skew, small batches, CPU scheduling, or KV pressure. Join expert statistics with collective, compute, and scheduler traces. If EPLB merely moves the wait elsewhere, remove it and fix the actual bottleneck.

Throughput improves but tail latency regresses

A rebalance or high-throughput dispatcher may favor batch efficiency while interactive requests wait. Separate prefill and decode metrics, examine the window around remaps, and test the low-latency or auto mode only as a new controlled experiment. A mean-throughput win does not override an explicit TTFT or TPOT service budget.

Results change after a rebalance

Stop promotion. Verify all ranks share the same logical-to-physical map, weight movement finished before use, quantized tensors and scales stayed paired, and graph capture references current buffers. Re-run deterministic token, parser, tool-call, streaming, and cancellation fixtures. Treat any unexplained semantic delta as a correctness failure, not performance noise.

Expert load changes by tenant or time of day

One global mapping may not serve every phase. Segment telemetry and compare whether a longer observation window, separate deployment pools, or static EP is safer than frequent remapping. Never use sensitive prompt text in the EPLB receipt; aggregate expert and rank metrics with the minimum tenant metadata needed for the decision.

Frequently asked questions

Does GLM-5.2 need expert parallelism?

Large self-hosted GLM-5.2 checkpoints generally need distributed placement, and EP is designed for its routed MoE weights. The exact TP, DP, and EP layout depends on checkpoint precision, accelerator memory, fabric, context, concurrency, and runtime. EP can be useful without EPLB.

Does GLM-5.2 need EPLB?

Not by default. Enable it only after static EP on a production-shaped workload shows persistent expert or rank skew and a canary proves an end-to-end benefit. If balancedness is already stable or memory is tight, zero redundant experts is the better outcome.

How many GLM-5.2 experts are active per token?

The pinned config routes each token to eight of 256 logical routed experts, or 3.125%. It also declares one shared expert. Top-8 does not mean only eight experts need to be resident because a later token can select a different set.

Is num_redundant_experts global or per rank in vLLM?

At the pinned source revision, it is a global count. EPLB builds 256 logical experts plus r redundant physical entries, then distributes that pool across EP ranks. The total must divide evenly. The documentation label is ambiguous, so pin and inspect the implementation again when upgrading.

How much BF16 memory does one redundant slot cost?

One extra slot on every rank represents 5.273438 GiB per rank across the 75 base MoE layers. The checkpoint-wide figure is 5.34375 GiB if the runtime also replicates its expert-bearing MTP block. Both exclude every non-routed-expert allocation and must not be used as full HBM requirements.

Should I set num_redundant_experts to 32?

Not automatically. It is a generic large-scale suggestion in the pinned vLLM documentation, not GLM-5.2-specific evidence. With 256 logical experts it is divisible for EP8, EP16, and EP32, but invalid for EP64, EP128, and EP256. Even where valid, it adds 12.5% routed-expert payload globally and still needs a measured workload benefit.

Which balancedness value is good enough?

There is no universal threshold. Use the mean-to-maximum load ratio as one trend, then require improvement in the declared end-to-end service metric over repeated windows without violating quality, memory, TTFT, TPOT, errors, or recovery. A ratio near one is not itself a promotion decision.

Can I use the same EPLB settings on vLLM and SGLang?

No. The runtimes expose different dispatchers, kernels, topology constraints, configuration semantics, and release boundaries. Reuse the measurement method, not unverified flags or values. Establish a separate pinned baseline and canary receipt in each runtime.

Sources and method

The model geometry comes from the pinned GLM-5.2 config, tensor index, and model card. Runtime semantics come from pinned vLLM expert-parallel docs, parallel configuration, EPLB state, and default policy.

The SGLang boundary uses its pinned GLM-5.2 cookbook and expert-parallel page. The algorithm reference is DeepSeek’s pinned EPLB repository. The fixed per-round Z.AI release check and Zhipu research check were discovery inputs, not EPLB performance evidence.

The committed evidence packet records the observed byte count and SHA-256 of 17 public responses. Its deterministic script checks 58,368 expert tensor names, derives the 75-base-layer and 76-checkpoint-block bounds, calculates six EP layouts, and tests the global r=32 divisibility case. The public JSON receipt is byte-identical to the committed result.

No checkpoint shard was downloaded. No container, model endpoint, grader, GLM, MiniMax, GPU, or NPU was used. The arithmetic is a BF16 routed-expert weight-only structure, not process fit. Current release observations and main-branch files are dated source receipts, not promises about future flags, defaults, platform support, or performance. The broad local deployment guide remains the place to choose between local hardware, a rented cluster, and an API; this page owns the EP/EPLB sizing and rollout decision.

About the editorial role

Evan Brooks

AI Infrastructure Editor

Evan Brooks serves as GLM52.ai’s disclosed editorial pen name for infrastructure, deployment, and training guides. The role examines checkpoint size, memory math, provider limits, implementation evidence, and operational risk. The GLM52.ai Editorial Team produces and fact-checks every article under this byline.

The GLM52.ai Editorial Team accepts responsibility for this article. Read our editorial policy or request a correction.

An AI image tool created this portrait. It does not depict a real individual.