GLM-5.2 Logprobs: Hosted and Local Score Contracts

Independent research — not an official Z.ai publication.Identity and provider disclosure

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial TeamPublished and editorially reviewed August 17, 2026

How we research and review Report a correction

Disclosed editorial pen name · AI-generated portrait

A cyan token stream crosses a locked gateway, fans into five probability paths, and ends at a local score display with an amber warning

Original editorial visualization of one token stream crossing a documented field boundary before local probability paths become observable. The amber marker represents the difference between token likelihood and correctness.

Use a self-hosted, version-pinned GLM-5.2 runtime when an application needs token-level scores. Do not add OpenAI-style logprob fields to the current Z.AI request and assume they work. Transformers 5.12 can return vocabulary scores during direct generation, vLLM 0.27.1 exposes chat and prompt score controls, and SGLang 0.5.17 exposes a native logprob contract. Their field names, defaults, response shapes, and score semantics are not one portable API.

Our dated audit hashes 12 public artifacts, evaluates 17 positive and negative profiles, and verifies 12 assertions. It also reproduces probability, ratio, joint-score, and length-normalized calculations. The audit makes zero authenticated API calls, zero tokenizer calls, and zero model calls. It proves the checked public contracts and arithmetic—not that a score predicts factual accuracy.

Where GLM-5.2 logprobs are available

The safest answer depends on the route, not only the model name:

Route checked	Output-token scores	Prompt-token scores	Checked control	Decision
Z.AI hosted Chat Completion	not documented	not documented	no logprob fields in the published request schema	omit the fields or use a separately documented route
Transformers 5.12 direct generation	available from generation scores	not a chat-API prompt-logprob feature	`return_dict_in_generate`, `output_scores`, then `compute_transition_scores`	use for controlled local analysis
vLLM 0.27.1 Chat Completions	`logprobs` plus `top_logprobs`	native extension `prompt_logprobs`	default maximum is 20 top scores	validate the server config and response dialect
SGLang 0.5.17 native generation	`return_logprob`	select a start with `logprob_start_len`	`top_logprobs_num` or chosen token IDs	treat the native envelope as a separate API

The pinned GLM-5.2 model card names Transformers 5.12.0, vLLM 0.23.0 or newer, and SGLang 0.5.13.post1 or newer as supported serving paths. That statement establishes model compatibility. It does not unify optional observability fields. This audit pins vLLM 0.27.1 and SGLang 0.5.17 so a later server change cannot be silently folded into the evidence.

The Z.AI schema does not publish logprob fields

The current Z.AI Chat Completion reference documents known fields such as model, messages, do_sample, temperature, and top_p. The same checked schema contains no documented request property named logprobs, top_logprobs, prompt_logprobs, logprob_token_ids, or logit_bias. The GLM-5.2 guide also contains no logprob term.

That is an absence finding, not a live rejection test. A proxy may accept an extension, ignore it, or return an error. This audit did not send a speculative request because an HTTP 200 would still not show that the backend applied the field. A production client should fail closed unless its exact provider, endpoint, and model route publish the request and response contract.

Keep the hosted request inside its documented envelope:

{
  "model": "glm-5.2",
  "messages": [
    { "role": "user", "content": "Classify this support ticket." }
  ],
  "do_sample": false,
  "max_tokens": 256
}

If token scores are a hard requirement, route that bounded workload to a local engine or another service that documents them. Do not let an adapter remove an unknown field and continue; the application would receive text without the evidence its decision gate expected.

One checkpoint, three local score interfaces

The pinned checkpoint’s generation_config.json contains neither logprobs nor output_scores. Score collection is therefore a runtime choice:

Transformers returns one vocabulary-sized score tensor per generated step when score output is enabled. A second method selects transition scores for the tokens that were actually generated.
vLLM’s OpenAI-compatible chat model uses logprobs: true and an integer top_logprobs. Its protocol also exposes prompt scoring and selected token IDs as extensions.
SGLang’s native GenerateReqInput uses return_logprob, a prompt start offset, a top-count, and an optional token-ID list.

These controls answer different questions. Output logprobs describe the generated path. Prompt logprobs score an existing token sequence under its preceding context. A selected-token list can compare a small label vocabulary without returning the full distribution. Record which task you need before choosing a field.

The values can also come from different stages. The checked vLLM model configuration defaults logprobs_mode to raw_logprobs and permits other raw or processed score modes. The Transformers source describes generated scores as processed vocabulary scores before softmax, then offers explicit normalization. Do not compare two columns merely because both are named logprob.

Convert logprobs without inventing confidence

For a normalized natural-log probability l, the token probability is:

p = exp(l)

The audit reproduces three anchors: exp(0) = 1, exp(ln 0.5) = 0.5, and exp(ln 0.1) = 0.1. A value closer to zero means the token had more probability mass under the recorded distribution. A lower value means less mass. It does not say why the model preferred the token.

A margin is often easier to interpret than one isolated value. If the chosen token has probability 0.72 and the runner-up has 0.18, their logprob difference implies a 4× probability ratio:

exp(log(0.72) - log(0.18)) = 4

That ratio concerns two next-token candidates. It is not a 72% chance that the completed answer is correct. Factual accuracy depends on the prompt, evidence, task, latent knowledge, later tokens, tools, and verification. The confidence-probability alignment paper studies when token probabilities and expressed confidence align; its existence is another reason to measure calibration instead of assuming it.

Joint likelihood shrinks with every token

Sequence log likelihood is the sum of token log probabilities. The equivalent joint probability is the product of token probabilities. For a three-token example with probabilities 0.9, 0.8, and 0.2:

sum logprob        = -1.9379419794
joint probability  = 0.9 × 0.8 × 0.2 = 0.144
mean logprob       = -0.6459806598
geometric mean     = exp(mean logprob) = 0.5241482788

Repeating the same three-token profile twice drops the joint probability to 0.020736 even though the per-token geometric mean stays 0.5241482788. Raw joint probability therefore penalizes longer candidates by construction. Compare fixed-length labels, or publish the exact length normalization used.

Tokenization changes the number and identity of scored units. Chat templates change the preceding context. Temperature, top P, repetition controls, and logit processors can change the distribution before or after a score is captured. Pin the tokenizer revision, rendered input IDs, chat template, sampling profile, and score stage before comparing runs. The chat-template audit owns the rendered-message boundary, while the sampling guide owns the stochastic-control baseline. If the experiment actively changes selected token logits, the GLM-5.2 logit-bias guide separates soft per-token steering from sequence bias and hard masks.

What the 17-fixture audit establishes

The zero-call validator exercises contract-positive, dependency, boundary, and type cases:

Fixture	Expected result	Why it matters
Z.AI request without score fields	accept	stays inside the published schema
Z.AI `logprobs: true`	reject locally	blocks an undocumented experiment
Transformers complete score path	accept	retains the score object and normalizes selected tokens
Transformers scores without return dictionary	reject	prevents a tensor-only result from hiding requested data
Transformers unnormalized scores	accept with warning	keeps raw versus normalized state visible
vLLM top five with `logprobs: true`	accept	follows the checked chat contract
vLLM top five without the flag	reject	enforces field dependency
vLLM top 21	reject under the default cap	catches a request above the checked default maximum of 20
vLLM prompt top five	accept	separates prompt scoring from output scoring
SGLang output-only scores	accept	preserves its default prompt start of −1
SGLang prompt start at zero	accept	makes prompt scoring explicit
SGLang score count without return flag	reject	avoids an inert native knob
SGLang prompt start −2	reject	catches an invalid local offset

The public machine-readable receipt contains all 17 fixtures, 12 source receipts plus hashes, the four probability calculations, and every assertion. It stores no source response body, key, cookie, prompt, generated answer, or model weight.

The fixtures do not prove that Z.AI rejects the fields, that a self-hosted cluster can fit GLM-5.2, or that one engine’s scores match another’s. They prove that the article’s route policy fails closed against the checked public contracts.

Reject score configuration drift before inference

Validate a score profile before allocating GPU memory or contacting a service:

const contracts = {
  zai: { fields: [] },
  transformers: {
    fields: ["return_dict_in_generate", "output_scores", "normalize_logits"],
  },
  vllm: {
    fields: ["logprobs", "top_logprobs", "prompt_logprobs", "logprob_token_ids"],
    maxTop: 20,
  },
  sglang: {
    fields: ["return_logprob", "logprob_start_len", "top_logprobs_num", "token_ids_logprob"],
  },
};

export function validateScoreProfile(route, profile) {
  const contract = contracts[route];
  if (!contract) throw new Error(`UNKNOWN_ROUTE:${route}`);
  for (const field of Object.keys(profile)) {
    if (!contract.fields.includes(field)) {
      throw new Error(`UNDOCUMENTED_OR_UNSUPPORTED:${field}`);
    }
  }
  if (route === "vllm" && profile.top_logprobs > 0 && profile.logprobs !== true) {
    throw new Error("TOP_LOGPROBS_REQUIRE_LOGPROBS_TRUE");
  }
  if (route === "vllm" && profile.top_logprobs > contract.maxTop) {
    throw new Error("ABOVE_CHECKED_DEFAULT_MAX_LOGPROBS");
  }
  if (route === "sglang" && profile.top_logprobs_num > 0 &&
      profile.return_logprob !== true) {
    throw new Error("SGLANG_SCORING_KNOBS_REQUIRE_RETURN_LOGPROB");
  }
  return { route, profile, checked: true };
}

Do not coerce a string to an integer, clamp 21 to 20, translate an SGLang field into a vLLM field, or drop an unsupported key. Return a configuration failure that names the route and checked version.

Collect normalized transition scores in Transformers

Direct generation needs both retained output metadata and per-step scores:

generated = model.generate(
    **inputs,
    max_new_tokens=32,
    do_sample=False,
    return_dict_in_generate=True,
    output_scores=True,
)

transition_logprobs = model.compute_transition_scores(
    generated.sequences,
    generated.scores,
    normalize_logits=True,
)

For a decoder-only model, remove the prompt length before pairing generated token IDs with transition_logprobs. Preserve the raw token IDs; decoded strings can merge spaces or byte pieces in ways that make audits harder.

The score tensor contains a vocabulary value for every generated step. Holding many long sequences can consume substantial memory. Bound output length and batch size, retain only required tokens, and measure peak memory on the actual engine. This code shape was not executed with GLM-5.2 weights in the audit.

Keep vLLM chat and prompt scoring separate

The checked vLLM chat protocol accepts an output-score request like:

{
  "model": "zai-org/GLM-5.2",
  "messages": [{ "role": "user", "content": "Return one label." }],
  "temperature": 0,
  "max_tokens": 8,
  "logprobs": true,
  "top_logprobs": 5
}

The default maximum of 20 comes from the pinned server model configuration and can be changed by an operator. prompt_logprobs and logprob_token_ids are useful native extensions, but a generic OpenAI SDK may require extra_body or may not expose them. Verify the serialized wire body, server version, response shape, and configured score mode.

When scoring a fixed label set, selected token IDs can avoid returning an unneeded top list. First confirm that each label maps to the intended token sequence under the pinned tokenizer. A human-readable label may begin with a space token or span several tokens.

Use SGLang’s native score envelope deliberately

SGLang’s pinned native input object makes the prompt boundary explicit:

{
  "text": "Classify: service unavailable\nLabel:",
  "sampling_params": {
    "temperature": 0,
    "max_new_tokens": 8
  },
  "return_logprob": true,
  "logprob_start_len": -1,
  "top_logprobs_num": 5,
  "return_text_in_logprobs": true
}

The default logprob_start_len=-1 means output-token scores only in the checked source. Starting at zero asks for prompt scoring as well. Selected token IDs use token_ids_logprob. Do not paste these names into a vLLM or Z.AI payload.

Native and OpenAI-compatible SGLang routes can expose different envelopes. Record the endpoint class as part of the contract. A server upgrade can also change response fields without changing the checkpoint revision.

Store score provenance before using a threshold

A useful score row needs more than token text:

{
  "model": "zai-org/GLM-5.2",
  "model_revision": "b4734de4facf877f85769a911abafc5283eab3d9",
  "runtime": "vllm",
  "runtime_version": "0.27.1",
  "endpoint_contract": "chat-completions",
  "score_mode": "raw_logprobs",
  "tokenizer_revision": "same-as-model",
  "prompt_digest": "sha256:REDACTED_EXAMPLE",
  "token_id": 101,
  "logprob": -0.328504,
  "rank": 1,
  "sampling_profile": "label-gate-v1"
}

Also store the rendered prompt-token digest, temperature, top P, penalties, reasoning mode, output cap, chat template, quantization, hardware class, and whether a score was raw or processed. Sensitive prompt text can remain outside the analytics table while a keyed or access-controlled digest ties the score to an evidence record.

Never merge rows from different tokenizers, templates, or score stages into one threshold fit. Re-run calibration after any component changes.

Calibrate on the task, not on fluent output

Build a labeled validation set that matches the decision:

Decision	Positive and negative evidence	Suitable metric	Unsafe shortcut
choose one fixed label	correct labels plus confusable labels	accuracy, coverage, risk at abstention threshold	highest token probability wins
accept extracted JSON	schema validity and field-level ground truth	exact match, field F1, abstention rate	average response logprob
route a support ticket	human-reviewed categories and escalation cases	class recall, cost, deferred share	fluent explanation
verify a fact	source-backed truth labels	factual precision and selective risk	token likelihood alone
detect generation drift	frozen prompts and runtime receipts	token-rank and score-distribution shift	comparing decoded text only

Fit a threshold on one split and report it on a held-out split. Include the coverage cost of abstention: a gate that rejects every case has low exposure but no utility. Track false acceptance separately from false rejection when their consequences differ.

Use deterministic verification where possible. A JSON parser, unit test, retrieval citation check, or policy rule can observe correctness more directly than token likelihood. Logprobs can support triage and drift detection; they should not replace task evidence.

Choose hosted generation or local observability

Choose Z.AI hosted access when standard documented generation fields meet the task and operating a large checkpoint would add cost without a required score signal. Choose a pinned local engine when prompt or output logprobs are part of a tested decision system and the deployment can meet the GLM-5.2 hardware requirements.

Local observability brings responsibility for GPU fit, score-mode selection, version pinning, tokenizer parity, privacy, capacity, and calibration. It does not make model probabilities truthful by default.

Dated logprob evidence and test boundary

Checked on August 17, 2026 (Hong Kong time). The source audit fetched the current Z.AI Chat Completion Markdown schema and GLM-5.2 guide, the Hugging Face model API, the pinned checkpoint README and generation config, Transformers 5.12 generation source, vLLM 0.27.1 chat protocol, vLLM’s pinned model configuration, SGLang 0.5.17 native input source, the calibration paper, the required Z.AI GLM-5.2 release entry, and the Zhipu research index. Each receipt records the URL, final URL, HTTP status, body byte count, and SHA-256; source bodies are not stored.

One exact Google result set for glm 5.2 logprobs token confidence was recorded as request serpapi-6be3c2cf5f0d41f0ace250b6b02b43d0. It returned generic confidence explanations, a provider logprob guide, the GLM-5.2 release, and a calibration paper, but no GLM-specific hosted-versus-local contract audit. That gap selected this narrow intent.

The validator and arithmetic ran in Node.js on public text. No GLM, MiniMax, grader, authenticated endpoint, credential, private prompt, tokenizer, model weight, GPU, or output-quality test was used. The audit cannot establish live Z.AI field rejection, provider implementation, cross-engine numeric parity, calibration, latency, memory fit, or factual accuracy.

GLM-5.2 logprobs FAQ

Does the Z.AI GLM-5.2 API support logprobs?

The Chat Completion schema and GLM-5.2 guide checked on August 17, 2026 do not document logprobs, top_logprobs, or prompt-logprob fields. That is not a live rejection test. Omit them unless the exact route publishes a separate extension contract.

Can Transformers return GLM-5.2 token probabilities?

Transformers 5.12 can retain per-step vocabulary scores with return_dict_in_generate=True and output_scores=True. Use compute_transition_scores(..., normalize_logits=True) to obtain normalized scores for selected generated tokens. This requires loading and serving the model; the audit did not do that.

What is the difference between logprobs and top_logprobs?

logprobs enables score output in the checked vLLM chat contract. top_logprobs selects how many high-probability alternatives to return at each generated position. The sampled token may also appear even when it falls outside the requested top count.

Can a logprob be used as a confidence score?

It can be a feature in a calibrated decision system. It is not a direct probability that the answer is correct. Validate the mapping on task-specific, held-out labels and report error plus abstention coverage.

Why do scores change after moving between engines?

Tokenizers, chat templates, score stage, sampling processors, quantization, hardware kernels, engine versions, and generated history can change the distribution. Preserve a full route receipt before comparing values.

Are prompt logprobs and output logprobs interchangeable?

No. Prompt logprobs score existing tokens under preceding context. Output logprobs describe generated token choices. They support different tasks and can use different response shapes.