GLM-5.2 Multiple Outputs: n vs Repeated Requests

Independent research — not an official Z.ai publication.Identity and provider disclosure

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial TeamPublished and editorially reviewed August 18, 2026

How we research and review Report a correction

Disclosed editorial pen name · AI-generated portrait

A cyan request node splits into four abstract candidate cards that pass through a shielded evaluation gate toward ranked output markers and a budget meter

Original editorial visualization of one request path becoming four candidate lanes, then crossing an evaluation gate. The independent pulses represent repeated requests; the shared fan-out represents a runtime-native count field.

Do not put n: 3 into every GLM-5.2 request. Repeat bounded calls on the documented Z.AI hosted route. Use num_return_sequences with an explicit sampling or beam strategy in direct Transformers generation. Use n only on the pinned vLLM or SGLang server contract that publishes it. In every route, reserve the full output budget, preserve candidate identity, and rank results outside the generation index.

Our dated audit hashes 15 public artifacts, resolves 24 valid and invalid plans, and passes 17 assertions. It makes zero authenticated API calls, zero tokenizer calls, zero model calls, and downloads no weights. It proves the checked source contracts and the planning guard—not provider acceptance, candidate diversity, throughput, cost, or answer quality.

Four routes, four count contracts

The checkpoint does not own the multi-output API. The runtime does:

Route checked	Published request control	Multiple-output constraint	Safe starting plan
Z.AI Chat Completion	no count field found	response has a `choices` list and result index, but the request schema does not promise several choices	repeat two or three documented requests
Transformers 5.12	`num_return_sequences`	sampled generation supports several returns; greedy without beams does not; returned sequences cannot exceed beams	`do_sample=True`, one beam, two or three returns
vLLM 0.27.1 OpenAI chat	`n`	`n` is validated and must equal one in greedy mode	sampled `n`, application cap, indexed accumulator
SGLang 0.5.17 OpenAI chat	`n`	protocol maps `n` into sampling parameters; its sampling guide discourages `n > 1`	repeat prompts unless a local benchmark justifies `n`

The pinned GLM-5.2 generation configuration sets temperature and top P but does not override num_return_sequences or num_beams. The model card names Transformers, vLLM, and SGLang as serving families; it does not define one portable candidate-count payload.

This page owns candidate fan-out and collection. The separate temperature and top P guide owns sampling- mode parity, and the max tokens guide owns route output limits. A multi-output plan must pass both decisions.

Repeat hosted Z.AI requests

The checked Z.AI Chat Completion reference documents do_sample, temperature, top_p, max_tokens, stream, and a response choices array. It does not document a request n, best_of, num_return_sequences, or num_beams field. That is an absence finding, not a live rejection test. A gateway could reject, strip, ignore, or privately implement an extra key; none is a safe public contract.

Generate hosted candidates as separately identified requests:

const candidateCount = 3;
const candidates = [];

for (let candidateIndex = 0; candidateIndex < candidateCount; candidateIndex += 1) {
  const response = await fetch("https://api.z.ai/api/paas/v4/chat/completions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.ZAI_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "glm-5.2",
      messages: [{ role: "user", content: "Propose one migration plan." }],
      do_sample: true,
      temperature: 0.8,
      top_p: 0.95,
      max_tokens: 512,
    }),
  });
  if (!response.ok) throw new Error(`candidate ${candidateIndex} failed`);
  const body = await response.json();
  candidates.push({ candidateIndex, requestId: body.request_id, choice: body.choices[0] });
}

Sequential calls make the bound visible and avoid an accidental concurrency burst. A production worker can use small controlled concurrency after rate, timeout, and cancellation tests. Record usage per request; do not multiply one response’s usage after the fact. A repeated prefix may qualify for the route’s prompt-caching behavior, but caching does not turn several provider requests into one request contract.

Treat retries as a separate state machine. A client timeout after submission does not prove that the provider failed to generate or bill the candidate. Store a local candidate ID before sending, attach the returned request ID after the response arrives, and mark an ambiguous attempt for reconciliation rather than issuing an automatic replacement. The documented request_id helps distinguish calls; this audit does not establish it as an idempotency key. When one attempt fails before a confirmed response, retain that failure beside the successful candidates. Replacing it silently would erase the route’s true latency, failure, and usage record and could exceed the declared count.

Decide what multiple means

“Give me three outputs” can describe three different jobs:

Sampled alternatives draw several continuations from a non-greedy token distribution. Use them for creative options or search over possible plans.
Beam hypotheses keep high-scoring partial sequences during one search. They are not independent attempts and can cluster around similar wording.
Repeated requests create distinct provider receipts, retries, latency, and usage records. They may share an identical prompt while remaining operationally separate calls.

Set the job before choosing the field. Temperature zero plus several slots does not establish useful diversity. vLLM rejects that combination; direct Transformers rejects several greedy returns unless beam search is active. SGLang’s schema can carry n, yet its own documentation recommends repeated prompts for control and efficiency.

No strategy decides which answer is correct. Candidate generation expands the search space. Validation and selection remain separate application stages.

Configure direct Transformers generation

Pinned Transformers 5.12 documents num_return_sequences and validates two important relationships in its GenerationConfig source: greedy generation without beams allows one return, while beam generation requires num_return_sequences <= num_beams.

For sampled alternatives from one prompt:

messages = [{"role": "user", "content": "Propose one migration plan."}]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

output_ids = model.generate(
    **inputs,
    do_sample=True,
    temperature=0.8,
    top_p=0.95,
    num_beams=1,
    num_return_sequences=3,
    max_new_tokens=512,
)
prompt_length = inputs["input_ids"].shape[1]
candidates = tokenizer.batch_decode(
    output_ids[:, prompt_length:],
    skip_special_tokens=True,
)

Transformers documents the final sequence tensor as batch_size × num_return_sequences rows in the pinned generation source. Preserve the original batch index when reshaping batched inputs; otherwise a candidate can be attached to the wrong prompt.

For two beam results, use do_sample=False, num_beams=4, and num_return_sequences=2. Beam search allocates work for every beam, not only the returned rows. If scores are required, request the generation result and sequence scores explicitly; do not infer score order from array position. The code above is a contract example, not a model run from this audit.

Treat vLLM n as a sampled output count

The pinned vLLM ChatCompletionRequest publishes n and passes it into SamplingParams. The pinned SamplingParams source requires an integer of at least one, applies a server-controlled maximum, and rejects n > 1 when temperature resolves to greedy sampling.

{
  "model": "<served-model-name>",
  "messages": [
    { "role": "user", "content": "Propose one migration plan." }
  ],
  "n": 3,
  "temperature": 0.8,
  "top_p": 0.95,
  "max_completion_tokens": 512,
  "stream": false
}

Do not use the engine’s maximum as the application default. Cap the user- visible request before it reaches the server; this audit uses four. The source allows the operator to change the server maximum, so a client that relies only on server rejection can change behavior after deployment.

The same source says asynchronous vLLM streams all n outputs cumulatively. An SSE parser must route every delta by candidate index rather than append all text into one buffer. The GLM-5.2 streaming guide covers reasoning, content, usage, finish reasons, and tool fragments; add a candidate dimension before enabling n > 1.

Prefer repeated prompts on SGLang

Pinned SGLang 0.5.17 also publishes n in its OpenAI chat protocol and maps the value into native sampling parameters. Yet the pinned sampling-parameter guide states that multiple outputs in one request are discouraged and that repeating the prompt offers better control and efficiency.

Honor that runtime-specific advice as the default. Submit a bounded batch of one-output prompts, preserve a candidate ID for each, and benchmark the exact server build before opting into n. If a local measurement supports n, retain an application cap and the same output-budget and stream-demultiplexing guards used for vLLM.

This distinction matters during engine migration. A field accepted by both servers can still have different scheduling, streaming, validation, and performance behavior. Schema similarity is not operational parity.

Reserve the whole candidate budget

Candidate count multiplies the worst-case output reservation:

reserved_output_tokens = candidate_count × per_candidate_max_tokens

Three candidates with a 512-token cap reserve 1,536 output tokens. Four with 8,192 reserve 32,768. Apply that multiplication before sending the request, even when a local engine shares prompt prefill work. It bounds memory pressure, queue occupancy, response size, and application exposure.

Hosted repeated requests report usage independently. Sum the returned receipts instead of predicting billing from a local formula. Local engines require a separate capacity test for KV cache and concurrency. Candidate count also multiplies downstream validation and ranking work, which should have their own timeouts.

Keep a request-level limit, a per-candidate token limit, a total output reservation, and a cancellation rule. Reject the plan when any candidate cap or total reservation is missing; silent clamping makes the receipt disagree with user intent.

Keep streamed candidates separate

For streaming n, key state by the response choice index:

const textByIndex = new Map();

for (const choice of event.choices ?? []) {
  if (!Number.isInteger(choice.index)) throw new Error("missing candidate index");
  const current = textByIndex.get(choice.index) ?? "";
  textByIndex.set(choice.index, current + (choice.delta?.content ?? ""));
}

Store reasoning, visible content, tool-call arguments, finish reason, and usage in separate fields for each candidate. Do not merge tool fragments from two indices. Do not mark the request complete until every expected candidate has a terminal reason or the enclosing request fails.

With repeated hosted requests, the outer request ID is the first identity key; the inner choice index is still worth retaining. An empty delta is not a new candidate, and stream order is not ranking order.

Choice index is not a quality score

An index identifies a slot. It does not prove that choice zero is the best, most likely, cheapest, or safest result. Define a selection policy that matches the application:

reject candidates that fail schema, safety, citation, or tool-argument validation;
score the survivors with a task-specific deterministic rubric when one is available;
retain the raw candidate, route, parameters, finish reason, token usage, validation result, and selected reason;
allow “no candidate passes” rather than forcing a winner.

Token log probabilities can support a narrow scoring analysis, but they are not calibrated correctness. The GLM-5.2 logprobs guide shows how route score envelopes differ. For structured tasks, validate the completed object using the structured-output boundary before ranking content.

Reject an invalid plan before inference

The audit’s portable guard uses a four-candidate policy cap and rejects plans before any model call:

function validatePlan(plan) {
  if (!Number.isInteger(plan.candidates) || plan.candidates < 1) {
    throw new Error("candidate-count-below-one");
  }
  if (plan.candidates > 4) throw new Error("portable-candidate-cap-exceeded");
  if (plan.candidates * plan.maxTokens > plan.totalOutputBudget) {
    throw new Error("candidate-output-budget-exceeded");
  }
  if (plan.stream && plan.candidates > 1 && !plan.candidateIndexAware) {
    throw new Error("candidate-index-required");
  }
  if (plan.selection === "index-zero" && plan.candidates > 1) {
    throw new Error("first-choice-is-not-ranking");
  }
}

Route-specific checks then reject Z.AI single-request-n, Transformers greedy multi-return, returns above beam count, and vLLM greedy n > 1. SGLang single-request-n is accepted only with a recorded warning because its audited documentation prefers repeated prompts.

The 24 fixtures include positive profiles for every route plus zero, string, and over-cap counts; total-budget overflow; index-blind streams; index-zero selection; invalid beams; greedy multi-output; and unaudited Transformers multi-sequence streaming. All 17 stored assertions pass.

Validate candidate receipts

A successful response should still pass these checks:

Receipt field	Gate	Failure meaning
expected candidate count	exact match, unless request failed atomically	missing or extra work
candidate identity	unique request ID and/or integer choice index	candidates cannot be reconstructed safely
visible output	preserve empty output with its finish reason	do not invent content during normalization
finish reason	retain per candidate	truncation, tool call, policy stop, or transport failure may differ
usage	store provider receipt as returned	candidate cost cannot be audited later
selection reason	explicit validation and ranking result	array position was mistaken for evidence

Test duplicate indices, out-of-order stream chunks, one truncated candidate, one failed repeated request, mixed tool and text choices, cancellation, and a budget rejection. Freeze the route, model identifier, source version, sampling profile, and prompt digest with the test corpus. Rerun after any client, runtime, model revision, or chat-template change.

Choose the route after the contract passes

Use repeated Z.AI requests when hosted operations, separate provider receipts, and published request fields meet the job. Use direct Transformers for a research harness that needs explicit sampled returns or beam hypotheses and can load the checkpoint. Use vLLM n when an OpenAI-compatible local endpoint, indexed collection, and engine cap are tested. Follow SGLang’s repeated-prompt default until the exact deployment supplies contrary measurements.

The local hardware guide covers the separate RAM, VRAM, storage, and deployment-fit decision. A valid n payload does not prove that the selected hardware can sustain the requested fan-out.

Dated source receipts and test boundary

Checked on August 18, 2026 (Hong Kong time). The audit fetched the current Z.AI Chat Completion schema and GLM-5.2 guide; the Hugging Face model API; pinned config, generation config, and README; Transformers 5.12 configuration and generation source; vLLM 0.27.1 chat and SamplingParams source; SGLang 0.5.17 chat, SamplingParams, and sampling documentation; the required Z.AI GLM-5.2 release entry; and the Zhipu research index. Receipts retain final URLs, statuses, byte counts, and SHA-256 hashes, not response bodies.

One exact Google result set for GLM-5.2 multiple outputs n num_return_sequences was recorded as request serpapi-0ae9eb8730684a86b7cb49b071788e33. The eight organic results contained the official release, generic GLM pages, and unrelated generalized linear model results; none explained GLM-5.2 candidate fan-out across these four contracts. The query was evaluated high-value because it confirmed a distinct supply gap and selected “multiple outputs” as the page wording.

The Node.js audit used public text and deterministic fixtures. It did not use an API key, call an authenticated endpoint, load a tokenizer, invoke GLM or MiniMax, download model weights, run a GPU, grade candidates, or measure live latency. It cannot prove undocumented hosted behavior, candidate independence, output quality, provider billing, engine throughput, memory fit, or future schema stability.

GLM-5.2 multiple output FAQ

Does the hosted Z.AI GLM-5.2 API support n?

The Chat Completion schema checked on August 18, 2026 does not document a request n or best_of field. Its response includes a choices list and result index, but that does not publish a request-side multi-candidate promise. Repeat bounded documented requests unless the exact route adds a count contract.

Why does vLLM reject n greater than one at temperature zero?

Pinned vLLM 0.27.1 treats near-zero temperature as greedy sampling and requires n=1 in that mode. Use a nonzero sampled profile for alternatives, or issue a single greedy result. Do not add slots that cannot establish diversity.

How do I return three sequences with Transformers?

For sampled candidates, use do_sample=True, num_beams=1, and num_return_sequences=3. For beam hypotheses, set num_beams to at least the number returned. Greedy generation without beams supports one returned sequence under the checked validation.

Is SGLang n the same as vLLM n?

Both checked chat schemas expose the field, but the operational contracts are not identical. SGLang’s own sampling guide discourages n > 1 and recommends repeated prompts. Preserve route-specific plans instead of translating by name alone.

Is choice zero the best candidate?

No. The index identifies the candidate. Apply explicit validation and a task- specific selection policy, store the reason, and allow every candidate to fail. Do not treat list position or stream arrival order as a quality score.

How many candidates should I request?

Start with two or three and impose an application cap. This audit uses four as a portable policy ceiling, not an engine maximum. Multiply count by the per- candidate output cap and reject plans that exceed the total output budget.