GLM-5.2 Stop Tokens: Audit EOS in vLLM and SGLang

Independent research — not an official Z.ai publication.Identity and provider disclosure

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial TeamPublished and editorially reviewed August 12, 2026

How we research and review Report a correction

Disclosed editorial pen name · AI-generated portrait

Generated token stream ending at three GLM-5.2 stop gates for end-of-text, user, and observation role boundaries

Original editorial diagram of the generated-token gate. The two upper rails represent prompt context; the lower stream represents output tokens. It is not a Z.ai, Hugging Face, vLLM, SGLang, RunPod, or GPU product screen.

GLM-5.2 does not declare one end-of-sequence token. At pinned official revision b4734de4facf877f85769a911abafc5283eab3d9, both config.json and generation_config.json list three EOS IDs: 154820, 154827, and 154829. The tokenizer maps them to <|endoftext|>, <|user|>, and <|observation|>.

That is an operational contract. A server that retains only 154820 can keep generating after a user or observation role marker. A diagnostic that enables ignore_eos can bypass every model-default stop. A home-grown scanner that searches the prompt can stop before generation begins because GLM-5.2 uses 154820 for left padding and embeds role tokens in valid prompts.

The safe starting profile is small: preserve all three checkpoint EOS IDs, leave ignore_eos false, inspect the effective request after every engine override, and test stop logic only against newly generated IDs. String cleanup and reasoning/tool parsing remain later, separate gates.

Read the three-token contract

The checkpoint and generation metadata agree exactly. Our probe fails if either file changes or if their ordered EOS lists diverge:

Token ID	Token text	Generated-token effect	Default?
`154820`	`<	endoftext	>`
`154827`	`<	user	>`
`154829`	`<	observation	>`
`154826`	`<	system	>`
`154828`	`<	assistant	>`

The descriptions for the two role-boundary stops are operational inferences from the IDs and tokenizer mapping. They are not a quotation about Z.ai’s training intent. The measurable claim is narrower: those token IDs are in the default EOS list, and the system and assistant role IDs are not.

Do not “normalize” this list to a scalar just because many causal models expose one EOS ID. Also do not add every role token. Adding 154826 or 154828 would create a new termination policy without checkpoint evidence. Start with the publisher metadata and make changes only through an explicit, tested request contract.

The pinned tokenizer also declares left padding and uses <|endoftext|> as both EOS and pad text. Sharing an ID is supported, but it makes the boundary between prompt inspection and output inspection important.

Separate prompt roles from output stops

The GLM-5.2 chat-template guide proves how messages become one prompt. This audit begins after that prompt enters the engine. Role markers in input are context; a matching token accepted from the decoder is a termination event.

Our prompt fixture deliberately contains two left-pad IDs, a user role ID, and ordinary content:

prompt:    [154820, 154820, 154827, 920, 921]
generated: [610, 611, 154820]

A naive scan reports a stop at prompt offset zero. The correct generated-only scan reports 154820 at output offset two. This is why substring searching an assembled transcript is a fragile substitute for the engine’s token-level finish reason.

Keep three layers distinct:

the template serializes roles, tools, and thinking controls into input IDs;
the decoder accepts new output IDs and stops when one matches its effective stop set; and
the reasoning/tool parser interprets the bounded output that remains.

A passing template fixture cannot prove layer two. A correct stop set cannot prove layer three. Debug the first layer where observed behavior diverges instead of patching the final string until the symptom disappears.

Verify the Transformers metadata path

Pinned Transformers v5.15.0 documents eos_token_id as either an integer or a list of integers, and GenerationConfig loads that field. The smallest checkpoint preflight does not need model weights:

import json

EXPECTED = [154820, 154827, 154829]

with open("config.json", encoding="utf-8") as handle:
    model_config = json.load(handle)
with open("generation_config.json", encoding="utf-8") as handle:
    generation_config = json.load(handle)

assert model_config["eos_token_id"] == EXPECTED
assert generation_config["eos_token_id"] == EXPECTED
assert model_config["pad_token_id"] == 154820
assert generation_config["pad_token_id"] == 154820

In an online loader, pin revision and inspect the resolved generation config before allocating accelerators. Record the revision, Transformers version, and effective EOS list beside the deployment. A friendly repository name is not an immutable input.

Request-level stop IDs can be additive. That does not make them a replacement for the model list. If a product adds a domain delimiter, test the union and retain the three defaults unless the engine’s documented override semantics say otherwise. Raw string stops are useful only for a separately specified text protocol; tokenization and boundary matching can make them behave differently from EOS IDs.

Audit vLLM’s effective stop set

At pinned vLLM v0.27.1, the request input processor supplies the tokenizer’s EOS ID and updates sampling parameters from generation-config fields. The sampling branch keeps 154820 as the primary EOS and adds 154827 plus 154829 to stop-token IDs. The resulting set is:

{
  "primary_eos_token_id": 154820,
  "stop_token_ids": [154827, 154829],
  "all_stop_token_ids": [154820, 154827, 154829],
  "ignore_eos": false
}

The model configuration defaults generation_config to auto. That is the expected path for importing repository generation defaults. The important override is --generation-config vllm: pinned source turns that option into an empty generation-config mapping. It can be valid when an operator wants vLLM defaults, but it is not evidence that the model’s supplemental EOS IDs survived.

Treat that flag as a review event. Inspect the effective sampling parameters or send deterministic boundary fixtures before promotion. Do the same after an engine upgrade, a custom generation-config file, an API gateway rewrite, or a request that supplies its own stop fields.

ignore_eos=true is an even stronger override. In the pinned branch it clears the primary EOS termination path; our translated diagnostic produced no active default stop. Use it only when a bounded test needs to observe output past EOS, with a strict token cap and isolated result. It should not be an unexplained production default.

Audit SGLang’s effective stop set

Pinned SGLang v0.5.17 unions EOS IDs from model and generation configuration into hf_eos_token_id, passes that set into each request, and checks newly accepted output tokens against multiple stop sources. With no request-specific additions, our translation is:

{
  "hf_eos_token_ids": [154820, 154827, 154829],
  "request_stop_token_ids": [],
  "tokenizer_eos_token_id": 154820,
  "additional_stop_token_ids": [],
  "effective_generated_stop_ids": [154820, 154827, 154829],
  "ignore_eos": false
}

The scheduling branch checks the request stop IDs, the Hugging Face EOS set, the tokenizer EOS, and tokenizer-defined additional stop IDs for accepted output tokens. The union matters more than which internal field first carries an ID. Your deployment receipt should therefore record the final set exposed to the request, not only the value in one JSON file.

SGLang also honors ignore_eos. The pinned default is false, but our negative control shows that true removes all three effective generated stops in the translated branch. A maximum-token limit still bounds a request; it does not restore correct semantic termination. Treat “always hits max tokens” as a configuration symptom, not proof that the model never emits EOS.

Test each generated boundary

The positive suite feeds the same short token pattern to both translated engine branches and changes only the intended boundary ID:

Fixture	Generated IDs	Expected match	vLLM	SGLang
end of text	`[610, 611, 154820, 612]`	`154820` at offset 2	pass	pass
next user boundary	`[610, 611, 154827, 612]`	`154827` at offset 2	pass	pass
observation boundary	`[610, 611, 154829, 612]`	`154829` at offset 2	pass	pass
non-stop roles	`[610, 154826, 154828, 612]`	no match	pass	pass

These fixtures are intentionally deterministic. They verify configuration and branch semantics without pretending to predict which token the 753B model will emit. A later live canary should add one synthetic conversation for ordinary text, one turn transition, and one tool-result transition, then assert finish reason, bounded output, and parser state.

Archive token IDs as well as decoded text. A parser may remove a special token before an application log sees it, while a gateway may expose a generic finish reason. The engine receipt is the most useful layer for deciding whether the decoder saw the expected boundary.

The sanitized machine-readable result is available at the public stop-token receipt. It preserves all twelve source hashes, versions, token fixtures, negative controls, 48 assertion outcomes, and the no-weights/no-server boundary.

Keep three negative controls

A test suite that only confirms the expected profile can pass while its failure detector is broken. Retain these controls:

Drop the supplemental role stops

Configure only [154820]. The user-boundary and observation-boundary fixtures must both fail to match. If they pass, the harness is importing an unrecorded stop source or checking decoded strings instead of the declared token set.

Enable `ignore_eos`

Feed [610, 154827, 154829, 154820] with ignore_eos=true. Neither translated engine branch should terminate on the model defaults. Bound the diagnostic by output tokens so the deliberate non-stop cannot run indefinitely.

Put stop IDs in the prompt

Left padding plus a user role marker must not end generation before the first new token. If a test reports that prompt match as a finish event, move the scanner to the decoder’s accepted-output path.

Do not “fix” these controls by adding three raw strings to every request. String stops may span different token boundaries, interact with detokenization, or truncate user-visible text that merely contains a marker-like sequence. Correct the metadata or engine configuration first; add application delimiters only when their protocol and tests require them.

Turn stop behavior into a release gate

Use one receipt per engine profile and fail closed when any row changes:

Layer	Required evidence	Stop condition
Checkpoint	pinned revision; model and generation EOS lists agree	digest or list changes without review
Tokenizer	five mapped role IDs; left padding; pad ID `154820`	role mapping or padding contract changes
Engine	version/commit and effective set are recorded	one of three defaults disappears
Request	`ignore_eos=false`; additions are explicit	gateway or client silently overrides stops
Positive tests	all three boundaries stop at exact generated offsets	continuation after any boundary
Negative tests	single-EOS, ignore-EOS, and prompt-scan controls fail as designed	harness cannot detect bad profiles
Parser	reasoning/tool layers receive one bounded assistant turn	stop pass is mistaken for parser pass
Runtime	output cap and cancellation remain active	semantic stop is the only safety bound

Roll out one engine version at a time. Keep the previous launch command and generation config as the rollback point. For a live tool canary, pair this gate with the site’s tested tool-calling loop. For transport truncation and terminal-event handling, use the streaming API parser guide.

Observe stop ID or finish reason by route, engine version, template revision, and request profile. Alert on a rise in maximum-token finishes, role markers in visible output, repeated assistant turns, or tool loops that wait for an observation after generation should have ended. Do not log private prompts to obtain this telemetry; synthetic fixtures and bounded counters are sufficient.

Diagnose stop-token failures by symptom

Symptom	First check	Likely boundary	Do not assume
`<	user	>` appears and text continues	effective set contains `154827`
tool loop generates an observation role	effective set contains `154829`	observation stop or parser integration missing	tool result itself is invalid
every request reaches `max_tokens`	`ignore_eos` and finish reason	EOS path bypassed or stops absent	model never emits EOS
request stops before decoding	prompt/output ID separation	prompt was scanned for stop IDs	pad and EOS cannot share an ID
system/assistant marker ends output	custom stop additions	unverified role IDs were added	all role tokens should be stops
output stops but tool parsing fails	parser flags and schema	layer three, after correct termination	more EOS IDs will fix parsing
one engine passes and another leaks	pinned versions and resolved sets	engine-specific import/override semantics	same CLI flags mean same behavior

When decoded output hides the matched special token, reproduce with a synthetic token fixture at the engine layer. If that layer passes, move outward to the gateway, SDK, stream assembler, and parser. Change one layer per test so a successful workaround does not erase the root cause.

Rent a canary only after the metadata gate

Stop-token validation itself is CPU-light. The evidence run needed no GPU and should happen before the larger capacity planning in the local GLM-5.2 hardware guide. Rent compute only when the pinned source, effective engine profile, and deterministic controls all pass and you need a live generation canary.

Audit the sources and evidence boundary

Sources checked on August 12, 2026:

the pinned GLM-5.2 model repository, model config, generation config, and tokenizer artifacts for exact IDs and mappings;
Transformers v5.15.0 generation configuration for scalar-or-list EOS metadata;
vLLM v0.27.1 sampling parameters, input processor, and model configuration for import, effective-set, override, and ignore_eos behavior; and
SGLang v0.5.17 model configuration, scheduler, and schedule batch for EOS union, request propagation, accepted-token checks, and bypass logic.

The Z.ai GLM-5.2 release confirms the model and supported open-weight serving context, but the stop behavior in this article is grounded in pinned artifacts and engine source. The probe’s simulations are dependency-free translations of only the cited branches, guarded by exact source hashes. We did not import or start the engines. A live release still needs the canary described above.

The full research record and probe scripts are versioned in the site’s private repository; the public JSON receipt is the reader-accessible audit. Re-run the source gate when any model revision, engine tag, generation config, or client override changes.

Answer common GLM-5.2 stop-token questions

What are GLM-5.2’s default EOS token IDs?

At pinned revision b4734de4…, both model and generation config list [154820, 154827, 154829]. They map to end of text, user role, and observation role tokens. Always recheck the revision you deploy.

Why are the user and observation tokens stops?

They are configured as EOS IDs. Operationally, stopping prevents one assistant generation from continuing into the next user or observation role. That description is inferred from the token map; it is not a claim about hidden training intent.

Should system and assistant role IDs also be stops?

Not by default. IDs 154826 and 154828 are absent from the pinned EOS list, and the non-stop fixture confirms they should not match this contract. Adding them requires a separate protocol and test.

Is `max_tokens` enough if EOS configuration is wrong?

It limits damage and cost but does not restore semantic turn boundaries. A request can leak role markers, produce malformed tool transitions, or waste its full output budget before the cap ends it.

Should I add the three token strings as stop strings?

Fix the engine’s token-ID configuration first. String stops operate after or during detokenization depending on the stack, can cross token boundaries, and may truncate marker-like text. Use them only for a defined application protocol with its own fixtures.

Does this prove vLLM and SGLang run GLM-5.2 correctly?

No. It proves pinned source and configuration branches plus deterministic stop fixtures. It does not start either engine, load weights, generate a model token, validate parsers, or measure performance.