Can GLM-5.2 Run on CPU in vLLM? A Sparse-Attention Audit

Evan Brooks

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial Team · Published and editorially reviewed August 30, 2026

How we research and review · Report a correction

Disclosed editorial pen name · AI-generated portrait

The default GLM-5.2 config does not enter vLLM CPU

At the pinned vLLM v0.28.0 and checked-main snapshots, a positive index_topk selects sparse attention, while the CPU platform explicitly rejects sparse attention. Deleting the key from config.json is not an opt-out: the pinned Transformers config supplies a typed default of 2,048. An index_topk=0 dense path exists only in an open, unmerged proposal and has not passed a full GLM-5.2 CPU startup or output-parity test.

Original editorial diagram. It explains the admission decision; it is not a vLLM screenshot, exception trace, CPU run, model-output sample, memory test, or performance benchmark.

Short answer: not with the default GLM-5.2 configuration on the pinned vLLM v0.28.0 release or the main-branch snapshot checked on August 30, 2026. GLM-5.2 declares index_topk=2048. Both runtime snapshots treat the presence of that field as a sparse-attention signal, pass the sparse flag into attention selection, and reach a CPU backend that raises because sparse attention is not supported there.

This is a runtime-admission answer, not a universal statement that CPUs can never execute GLM-5.2 mathematics. An open vLLM pull request proposes treating index_topk=0 as an explicit dense-MLA fallback. That proposal has useful config tests, but it is unmerged and reports no full Linux CPU startup or output-parity result for the complete checkpoint. It is therefore a hold, not a supported workaround.

Our machine-readable CPU support audit hashes 26 official or upstream source receipts. It made zero model calls, downloaded zero model-weight bytes, imported vLLM and Transformers zero times, ran zero CPU or GPU inference jobs, started zero serving processes, and ran zero containers. The audit proves a pinned source path and proposal state. It cannot prove full-model CPU correctness, output parity, memory fit, acceptable speed, or a future merge.

In this guide

1. Read the pinned verdict
2. Trace the current admission path
3. See why deleting index_topk does not work
4. Separate stable, main, and the open proposal
5. Do not confuse dense MLA plumbing with GLM support
6. Distinguish functional support from practical deployment
7. Inspect installed source without importing vLLM
8. Apply the fail-closed CPU canary gate
9. Avoid local self.is_v32 patches
10. Design the full-model acceptance run
11. Choose a supported route while the gate is closed
12. Frequently asked questions
13. Bottom line
14. Sources and method

Read the pinned verdict

Four separate questions determine whether “GLM-5.2 on CPU” is an honest claim. The first two fail in the audited runtime; the last two have not yet been tested for the proposed fallback.

Question	Pinned evidence on August 30, 2026	Decision
Does the checkpoint request sparse attention?	The immutable GLM-5.2-FP8 config declares index_topk=2048	Yes
Can the audited vLLM CPU backend accept that path?	v0.28.0 and checked main both contain the explicit sparse-attention CPU rejection	No
Is a dense opt-out merged into a release or pinned main?	PR #54029 proposes zero as dense, but remains open and unmerged	No
Has the complete checkpoint passed CPU startup and output parity on that opt-out?	The proposal reports config tests, not a full GLM-5.2 Linux CPU run	No

The current verdict is therefore reject the default CPU launch and hold the dense fallback. That is more precise than saying “vLLM supports CPU” or “GLM-5.2 can fit in enough RAM.” A framework can support a device class while a particular architecture requires an unsupported attention backend.

This page owns that compatibility gate. The local hardware guide owns checkpoint size, RAM, VRAM, context memory and cost. The vLLM compile audit owns torch.compile eligibility and eager fallback. The AMD SGLang guide owns ROCm accelerator deployment. Passing one contract does not satisfy another.

Trace the current admission path

The failure is a short, inspectable chain rather than a vague hardware incompatibility.

1. The pinned GLM-5.2-FP8 configuration names GlmMoeDsaForCausalLM, 78 hidden layers, 256 routed experts, eight active experts per token, 32 index heads, and index_topk=2048.
2. In the vLLM v0.28.0 model source, two locations set the GLM/DSA branch from whether the config has the index_topk attribute.
3. The attention wrapper receives that branch as its sparse flag.
4. The pinned v0.28.0 CPU platform checks the sparse flag first and raises the six-word error, “Sparse Attention is not supported on CPU.”

The checked-main snapshot at 680e2177e473ed8dfaa9773f7ead185b369cab46 preserves the same decisive structure: two attribute-presence checks, the sparse flag, and the CPU rejection. Its general CPU MLA implementation has continued to evolve, but the positive GLM index path still does not become a CPU sparse backend.

This chain is enough to block an expensive launch before weights are staged. It is not enough to say what a future dense implementation will output. Source reachability and model correctness are different evidence classes.

See why deleting index_topk does not work

Editing raw JSON looks plausible because stable vLLM asks whether the loaded config object has index_topk. The missing step is typed configuration.

At the pinned Transformers 5.16.1 GlmMoeDsaConfig, index_topk is a typed field with a default of 2,048. Removing the JSON member does not establish that the constructed object lacks the attribute; the class can restore its default during deserialization.

That gives three distinct values with different meanings:

Raw input	Effective typed value in the audited contract	Safe interpretation
Field present as 2048	2048	Sparse DSA requested; current CPU path rejects
Field deleted	2048 default	Still sparse; deletion is not an opt-out
Field explicitly 0	0	Only the open PR proposes interpreting this as dense MLA

Do not mutate a downloaded checkpoint in place. It breaks the immutable model receipt, makes later incident review ambiguous, and still does not change the stable runtime’s attribute-presence test. If a future merged release supports a documented override, keep the original checkpoint read-only and record the override separately in the launch manifest.

Also distinguish “zero is accepted by a parser” from “zero produces correct model behavior.” The open proposal changes code semantics. A config unit test can prove the branch selection, but only a full-model run can test attention, generation and application behavior.

Separate stable, main, and the open proposal

Do not call all three “latest vLLM.” They carry different evidence and should produce different operational decisions.

Snapshot	Immutable identity	CPU sparse gate	Dense zero semantics	Validation state
vLLM v0.28.0	2cf0a6915ce544dc493a0990f2ea38d81601128a	Rejects	Absent	Released source; default GLM config is blocked
Checked main	680e2177e473ed8dfaa9773f7ead185b369cab46	Rejects	Absent	Later source; same decisive gap
Open PR #54029	3d6db6b56381dc26d595509683fb556c24591e5a	Adds earlier actionable error	Proposes index_topk=0 as dense	Open, unmerged, zero captured approvals, no full-model parity

PR #54029 changes three files: the GLM config verifier, the two model checks, and config tests. The proposal replaces attribute presence with a helper that treats a nonzero value as sparse. On CPU, it would instruct an operator to pass an explicit zero override. Its tests cover zero and 2,048, and the author reports 17 matching config tests passed.

Those are useful receipts, but they stop at configuration. The PR explicitly records that a full GLM-5.2 Linux CPU startup and output evaluation were not run. The captured review set has no approval, and the PR is not merged. Never build a production promise from a proposed diff’s green local unit test.

If the PR later merges, pin the containing commit or release and re-read the actual merged files. Review feedback can alter semantics, tests or the CLI contract. A PR number is not an immutable runtime identity.

Do not confuse dense MLA plumbing with GLM support

There is a nearby positive signal that can be misunderstood. Merged vLLM PR #51471 adds CPU-native MLA prefill pieces, contextful cache handling and tests. That is real upstream progress, and it explains why a dense GLM fallback is technically discussable.

Its stated validation target is DeepSeek-V2-Lite, including dummy weights and external KV-hit paths. It does not add the GLM DSA index_topk=0 opt-out. The audit therefore records two separate capabilities:

• CPU MLA plumbing: merged for the documented DeepSeek-oriented path;
• GLM-5.2 DSA admission: still blocked by the positive sparse index gate in stable and checked main.

Architecture families can share attention scaffolding without sharing every indexer, sparse kernel, cache layout or config contract. Treating a generic MLA merge as model support would skip the exact branch that fails here.

The official v0.28.0 CPU documentation lists x86, ARM, Apple Silicon and IBM Z variants. Its validated CPU model matrix includes a smaller GLM-4 9B checkpoint but not GLM-5.2. General platform documentation is not a model-specific validation matrix.

Distinguish functional support from practical deployment

Even a future correct dense CPU path would answer only “can it execute?” It would not answer “should this production workload use it?”

The local hardware guide calculates checkpoint and context fit separately for BF16, FP8 and quantized routes. A functional CPU canary still needs enough memory for weights, runtime state, KV cache, temporary buffers, the operating system and failure headroom. Dense attention can also change the compute profile relative to the sparse design.

Use a two-axis decision:

Axis	Minimum receipt	What it still does not prove
Functional compatibility	Merged pin, effective config, full startup, deterministic output and failure parity	Useful latency, throughput, concurrency or cost
Operational practicality	Measured memory high-water mark, prefill/decode latency, throughput, energy and spend on the target host	Semantic correctness unless the functional gate also passed

A tiny or truncated dummy model can help exercise configuration, but it cannot stand in for all 78 layers, real weights, sparse/dense attention numerics, long-context state, expert routing or memory pressure. Label every reduced fixture as a fixture.

Likewise, Apple Silicon support in vLLM v0.28.0 is documented as experimental and requires a source build. That platform status is separate from the architecture gate. Moving the same default config from an M3 to a large x86 server does not manufacture a CPU sparse backend.

Inspect installed source without importing vLLM

Before staging weights, hash and inspect the files that will execute. This script uses package metadata to locate source without importing vLLM or initializing a runtime:

Audit the installed GLM CPU gate without importing vLLM

from hashlib import sha256
from importlib.metadata import distribution, version
from pathlib import Path

dist = distribution("vllm")
paths = {
    "cpu": "vllm/platforms/cpu.py",
    "model": "vllm/model_executor/models/deepseek_v2.py",
    "verify": "vllm/model_executor/models/config.py",
}

print("vllm", version("vllm"))
texts = {}
for name, relative in paths.items():
    path = Path(dist.locate_file(relative)).resolve()
    data = path.read_bytes()
    texts[name] = data.decode(errors="replace")
    print(name, len(data), sha256(data).hexdigest())

print({
    "cpu_rejects_sparse": "Sparse Attention is not supported on CPU" in texts["cpu"],
    "presence_checks": texts["model"].count('hasattr(config, "index_topk")'),
    "zero_helper_present": "_is_sparse_mla_enabled" in texts["verify"],
    "actionable_override_present": "--hf-overrides" in texts["verify"],
})

Archive the output beside the wheel hash, container digest if applicable, checkpoint revision, Transformers version, raw config hash and sanitized launch manifest. Compare it to the source revision you reviewed. Version text alone does not prove that a wheel was not rebuilt or patched.

Inspect the effective typed config separately. Do that only in the controlled future validation environment, because constructing model configuration may import dependencies. Record model_type, architecture and index_topk after loading. Reject an unexpected value or a runtime whose zero semantics are not present in the exact pinned source.

Apply the fail-closed CPU canary gate

The public receipt evaluates four deterministic policy fixtures. They test the decision rule, not GLM-5.2 execution.

Fixture	Effective index_topk	Zero-aware merged runtime	Full startup + parity	Decision
v0.28.0 default	2048	No	No	Reject: sparse CPU backend unavailable
JSON key deleted	2048 typed default	No	No	Reject: deletion did not disable sparse routing
Open PR override	0	Not merged	No	Hold: proposed branch only
Future merged and validated override	0	Yes	Yes	Eligible for a bounded functional canary

Encode the same state machine in the release gate:

effective index_topk > 0 on CPU        -> REJECT
index_topk = 0, semantics not merged   -> HOLD
merged zero semantics, parity missing  -> HOLD
merged pin + full startup + parity      -> FUNCTIONAL-CANARY-ELIGIBLE

“Eligible” deliberately does not mean production-ready. It authorizes a bounded functional canary whose resources, time, data, logging and cleanup are named in advance. Hardware practicality is evaluated afterward.

Fail closed if the installed-source hash differs, the effective config cannot be captured, the model is truncated, the startup receipt belongs to another process, any output field diverges, or the proposed code is still unmerged. Do not convert an unknown state into a pass by repeating launches.

Avoid local self.is_v32 patches

Issue #54018 reports that setting the local model branch to dense allowed a truncated, dummy-weight fixture to run. That is a useful diagnostic observation, not a safe deployment recipe.

A local edit changes at least four contracts:

• which attention path executes;
• whether indexer state and buffers are constructed;
• the provenance of the installed runtime;
• the set of upstream tests and review assumptions that apply.

Process health cannot validate those contracts. Dummy weights cannot validate language behavior. A three-layer fixture cannot validate all 78 layers. A short prompt cannot validate long-context cache state or role-aware stop tokens.

Do not patch self.is_v32, monkey-patch the platform gate, or maintain a private wheel merely to make startup proceed. If development work is required, keep it isolated as a clearly labeled experiment, retain an unmodified control, and do not expose it as a supported service. This guide does not authorize or validate such a fork.

Design the full-model acceptance run

After a dense opt-out is merged, prepare the acceptance corpus before renting or allocating hardware. Pin every input: checkpoint revision, merged vLLM commit or release, Transformers revision, CPU architecture, OS, PyTorch build, dtype, thread binding, NUMA layout, memory limits and launch overrides.

The functional suite should include:

1. a full, untruncated checkpoint startup with real weights;
2. deterministic short-answer and exact-token fixtures;
3. thinking-on and thinking-off chat-template paths;
4. all three role-aware stop-token outcomes;
5. single and multi-turn tool calls, including denied and malformed arguments;
6. streaming and non-streaming response reconstruction;
7. named long-context points with cache and memory receipts;
8. concurrency, cancellation, timeout, OOM and restart behavior;
9. parity against a known-good supported runtime with tolerances fixed in advance;
10. immediate rollback to the immutable control.

Compare token IDs where policy permits, decoded content, reasoning/final-answer boundaries, tool names and arguments, finish reasons, usage counts, error classes and application acceptance. A dense fallback may be mathematically intended to preserve results, but the acceptance test must establish that for the pinned implementation and workload.

Only after correctness passes should you measure prefill time, inter-token latency, throughput, peak resident memory, swap activity, concurrency and energy or rental cost. Keep those metrics out of the correctness verdict.

Choose a supported route while the gate is closed

If the goal is functional verification today, use a checkpoint and runtime combination with explicit accelerator support rather than turning the CPU gap into an unreviewed source fork. The official GLM-5.2 vLLM recipe is a starting point, not a guarantee for every GPU, topology or workload.

Define the GPU model and count, memory, interconnect, driver, CUDA, vLLM image, checkpoint pin, storage, egress, spend ceiling, test duration, data policy, acceptance corpus and deletion plan before provisioning. A rentable SKU does not prove that its software stack matches the recipe.

Use a supported GPU path while the vLLM CPU gate remains closed

Use this route only after the hardware-fit plan, exact runtime and checkpoint pins, correctness corpus, spend ceiling and deletion procedure are written. Keep the CPU experiment separate; renting a GPU does not validate the proposed dense fallback.

Check current GPU availability on RunPod →

Affiliate disclosure: This is an affiliate link and a referral link. GLM52.ai may receive a commission or referral credit after qualifying usage at no added charge from us. A rental does not prove GLM-5.2 compatibility, sufficient GPU count or memory, correct interconnect, driver, CUDA, vLLM, checkpoint, storage, egress, price, security, data handling, output parity, or cleanup. Verify the live SKU, immutable source and model pins, official recipe boundary, acceptance results, terms, budget and rollback before use.

If the workload does not justify accelerator provisioning, use a hosted API or a smaller validated model instead. The correct fallback depends on privacy, latency, cost and control requirements; the access-route guide separates subscription, metered API and self-hosting decisions.

Frequently asked questions

Does vLLM support CPUs generally?

Yes. The pinned documentation covers x86, ARM, Apple Silicon and IBM Z CPU variants. That platform support does not mean every model architecture is validated. GLM-5.2’s positive sparse-index path reaches an explicit CPU backend rejection in the two audited runtime snapshots.

Will removing index_topk from config.json fix GLM-5.2 CPU startup?

No evidence here supports that. The pinned Transformers class supplies a typed default of 2,048, so omission does not remove the attribute. It also breaks the immutable checkpoint receipt. Keep the original config and reject the launch.

Can I pass index_topk=0 now?

Not as a supported v0.28.0 or checked-main solution. Zero-as-dense is proposed in open PR #54029. It is unmerged and lacks full-model startup and parity evidence. Wait for a merged pin, then re-audit and test it.

Did the merged CPU MLA pull request fix GLM-5.2?

No. PR #51471 adds important CPU MLA functionality for its documented DeepSeek-V2-Lite path. Its scope does not provide the GLM DSA sparse opt-out that the current default configuration needs.

Is this failure unique to Apple Silicon?

The report used an M3, and Apple Silicon CPU support is experimental. However, the audited CPU platform code expresses the sparse rejection at the CPU device class level. This static audit did not run x86, ARM, Apple Silicon or IBM Z and does not claim identical secondary failures on each architecture.

Would a successful CPU startup prove the deployment is practical?

No. Startup is only one functional receipt. Full output parity, failure behavior, memory, latency, throughput, concurrency and cost remain separate. The checkpoint’s hardware fit must be measured on the actual host.

Does this page prove the reporter’s local dense patch is correct?

No. It verifies that the pinned source contains the reported admission mechanism. It did not execute the patch, load weights, import vLLM, or run a model. Reporter observations stay attributed to the issue.

Bottom line

The pinned answer is narrow and actionable: default GLM-5.2 cannot enter the audited vLLM CPU backend because its effective positive index_topk selects a sparse path that CPU explicitly rejects. JSON deletion does not remove the typed default. The open zero-as-dense proposal is a promising branch contract, not merged support and not model-correctness evidence.

Keep the current CPU launch closed. Pin and hash the source, wait for a merged fix, inspect the effective config, run the complete checkpoint, require output and failure parity, and only then authorize a bounded functional canary. Evaluate memory, speed and cost afterward.

Sources and method

Primary and upstream sources checked on August 30, 2026:

• Z.AI GLM-5.2 release page and the Zhipu AI research index, used as required discovery and model-context checks rather than CPU support proof;
• the immutable GLM-5.2-FP8 config and pinned Transformers GLM DSA config;
• vLLM issue #54018, open PR #54029, and merged CPU MLA PR #51471;
• pinned vLLM v0.28.0 and checked-main CPU, model and config-verifier source;
• the v0.28.0 CPU installation guide, Apple Silicon notes, validated CPU model matrix, and the pinned GLM-5.2 recipe.

The reproducible evidence bundle is archived in the project repository. Its public sanitized result is available as JSON, SHA-256 c62e51aaa378497de5b876e7f3df552b630041e5576fef9b2fcdafcfa83d0882. The static audit fetched and hashed 26 documents. It did not download weights, import a runtime, start a model, run CPU or GPU inference, or validate the open proposal’s output.

About the editorial role

Evan Brooks

AI Infrastructure Editor

Evan Brooks serves as GLM52.ai’s disclosed editorial pen name for infrastructure, deployment, and training guides. The role examines checkpoint size, memory math, provider limits, implementation evidence, and operational risk. The GLM52.ai Editorial Team produces and fact-checks every article under this byline.

The GLM52.ai Editorial Team accepts responsibility for this article. Read our editorial policy or request a correction.

An AI image tool created this portrait. It does not depict a real individual.