GLM-5.2 Offline Batch Inference with vLLM

Evan Brooks

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial Team · Published and editorially reviewed August 28, 2026

How we research and review · Report a correction

Disclosed editorial pen name · AI-generated portrait

run-batch is a file runner, not a durable Batch API

vLLM 0.23.0 can read OpenAI-style JSONL and execute the requests without a separate inference server. Its own documentation explicitly says this is not the complete Batch REST API. The pinned source submits every row to one local engine, waits for the gathered results, and then opens the output path in write mode. Validate the file before expensive model startup, use a new output path, split large jobs into bounded chunks, and treat a missing final receipt as an unfinished chunk—not permission to rerun unknown work blindly.

Original editorial diagram. It explains the intended reconciliation gate; it is not a vLLM screenshot, GPU run, model output, throughput measurement, or proof that any depicted request succeeded.

GLM-5.2 offline batch inference is useful when a finite set of prompts should consume one already-provisioned engine without operating an HTTP service. The safe unit of work is not “a folder of prompts.” It is a versioned JSONL input, a pinned runtime and checkpoint, a fresh output path, and a reconciliation receipt that accounts for every custom_id.

This guide uses the official FP8 checkpoint because the current vLLM GLM-5.2 recipe names zai-org/GLM-5.2-FP8 as its default variant and gives vLLM 0.23.0 as the minimum stable version. That is a very large deployment: the recipe reports an 893 GB minimum VRAM figure for the FP8 variant, while the pinned repository is 755.7 GB on disk. Complete the GLM-5.2 FP8 download and shard-verification gate before letting the engine read those weights.

Our machine-readable offline-batch audit hashes 17 official source snapshots, validates four request rows, rejects seven deliberate input failures, and reconciles four shuffled synthetic outputs. Two synthetic rows pass; an HTTP error and a finish_reason="length" row are quarantined. The audit made zero model calls, started zero vLLM processes, downloaded zero model-weight bytes, and ran on no GPU. Its results test the file contract and release logic, not GLM-5.2 generation.

In this guide

1. Choose offline batch only when the job fits
2. Freeze runtime and model coordinates
3. Treat JSONL as a release artifact
4. Make thinking policy explicit per request
5. Preflight seven failures before GPU startup
6. Set batch and memory caps intentionally
7. Run a small pinned batch
8. Expect concurrency, not a durable queue
9. Join every result by custom ID
10. Quarantine errors and length truncation
11. Reconcile usage and file receipts
12. Make reproducibility an acceptance test
13. Resume with chunk manifests
14. Know when a server or hosted API is better
15. Rent only after the fixture gate
16. Frequently asked questions
17. Sources and method

Choose offline batch only when the job fits

vLLM describes its LLM class as the primary Python interface for inference without a separate server. The vllm run-batch command adds a file-oriented OpenAI-compatible layer: each JSONL line looks like one HTTP request, but the runner invokes the local engine directly. That is a good match for bounded evaluation, classification, extraction, code review, or content-processing jobs whose complete request set can be frozen first.

It is not automatically the best route for every “batch” workload.

Route	Best fit	What it gives you	What you must build or verify
vllm run-batch	Finite local job on one pinned engine	No separate HTTP server; JSONL in and JSONL out	Preflight, chunking, crash boundary, output audit, and capacity controls
vLLM OpenAI-compatible server	Ongoing or multi-client traffic	Request-time admission, client retries, health checks, and familiar HTTP calls	Service security, queueing, rate limits, observability, and idempotency
Direct LLM.generate() or LLM.chat()	Python application that owns prompt construction	Typed in-process control and direct result objects	Your own input schema, persistence, scheduling, and receipt format
Hosted asynchronous batch product	Work that should survive client or node loss	Provider-managed storage, status, quotas, and retry semantics when documented	Data boundary, price, retention, supported model/fields, and provider-specific result rules

The distinction matters because “uses OpenAI batch file format” does not mean “implements OpenAI’s complete Batch API.” The vLLM 0.23.0 file-format guide says so directly. There is no job-creation endpoint, provider-side lifecycle, 24-hour window, or durable status object implied by this local command.

Choose the local runner when you can answer four questions before startup:

1. What exact request IDs belong to this chunk?
2. What immutable model and engine revisions will execute it?
3. Which result conditions are safe to release automatically?
4. What evidence proves the chunk is complete after a crash or operator handoff?

If any answer depends on live traffic or an external consumer, a server or a real durable workflow is usually easier to operate honestly.

Freeze runtime and model coordinates

The audited coordinates are:

Pinned offline-batch coordinates

vLLM release:       0.23.0
vLLM commit:        0fc695fc6d1d82e9a5ac6835ac8e4e1c83703665
Recipe commit:      5943215a27acb4a243e9d27bd69daf491034cfea
Model:              zai-org/GLM-5.2-FP8
FP8 model revision: ba978f7d347eaf65d22f1a86833408afdb953541
Base artifacts:     b4734de4facf877f85769a911abafc5283eab3d9

There is a documentation boundary worth preserving. The model table shipped at the vLLM 0.23.0 tag does not contain GlmMoeDsaForCausalLM or GLM-5.2. The official recipe nevertheless names 0.23.0 as the stable minimum, and the vLLM model table at audited main commit 5f213ed1592903b7bc38f173d320dac1b2769303 now lists the architecture for GLM-5, GLM-5.1, and GLM-5.2. Do not compress those facts into “the 0.23 model table proves support.” It did not. Pin the recipe, engine build, model revision, and a real startup canary together.

The pinned FP8 config reports GlmMoeDsaForCausalLM, 78 hidden layers, 1,048,576 maximum positions, and block FP8 (e4m3) quantization. Those fields describe the artifact. They do not guarantee that your chosen kernel, topology, context cap, or speculative-decoding path is correct.

Record the complete software environment beside every input manifest:

Write immutable coordinates into the batch receipt

export GLM52_BATCH_MODEL='zai-org/GLM-5.2-FP8'
export GLM52_BATCH_REV='ba978f7d347eaf65d22f1a86833408afdb953541'
export GLM52_BATCH_DIR='/srv/glm52-batches/job-2026-08-28-001'

vllm --version
sha256sum "$GLM52_BATCH_DIR/requests.jsonl"

Also record the container digest or environment lock, driver, CUDA or ROCm stack, accelerator topology, engine arguments, and checkpoint-directory receipt. A package version alone is not a runtime identity.

Treat JSONL as a release artifact

The upstream format is one JSON object per line. For chat generation, every row needs a developer-selected custom_id, method, relative url, and request body. The body uses the chat-completions schema.

One non-thinking GLM-5.2 request line

{"custom_id":"extract-0001","method":"POST","url":"/v1/chat/completions","body":{"model":"zai-org/GLM-5.2-FP8","messages":[{"role":"system","content":"Return exactly one JSON object with keys risk and evidence."},{"role":"user","content":"Classify this fixture: the migration has no rollback test."}],"temperature":0,"top_p":1,"max_completion_tokens":128,"chat_template_kwargs":{"enable_thinking":false}}}

Keep each physical JSONL line complete; pretty-printed multi-line objects are not JSONL records. Use UTF-8, a final newline, and no comments. Generate the file from structured data rather than interpolating untrusted prompt text into a hand-built JSON string. A newline inside message content must be JSON-escaped as \n, not emitted as a new record.

custom_id is the durable join key. Make it unique across the entire logical job, not only within one file. A useful ID carries a task family and stable sequence or source key—review-20260828-000042 is easier to investigate than request-42. Do not put an email address, secret, customer name, or raw prompt inside the ID because it will appear in input, output, logs, and receipts.

Our four-line fixture has these properties:

Fixture field	Audited value	Release purpose
Request rows	4	Small enough to inspect manually
Unique custom_id values	4	One-to-one reconciliation
Explicit non-thinking rows	2	Bounded extraction and classification
Explicit reasoning rows	2	Review and planning canary
Sum of completion ceilings	1,472 tokens	Upper-bound planning before prompt tokens
Canonical JSONL size	1,630 bytes	Detect accidental file change
Input SHA-256	85fda021…b63b865d	Tie runtime receipt to exact bytes

The completion-ceiling sum is not a usage forecast. A request may stop early, fail before generation, or consume its entire limit. It is simply the maximum configured completion allocation across this fixture.

Make thinking policy explicit per request

The pinned GLM-5.2 chat template selects high only when that exact reasoning_effort is provided; otherwise its effective reasoning effort is max. It also has an explicit non-thinking branch when enable_thinking=false. Therefore, omitting both fields is a behavior choice: it silently accepts the template default.

For an auditable batch, choose one policy per row:

Bounded reasoning row

"chat_template_kwargs": {"reasoning_effort": "high"}

or:

Non-thinking row

"chat_template_kwargs": {"enable_thinking": false}

Do not set both on the same row. A non-thinking extraction and a high-effort review are different acceptance profiles; mixing them accidentally can change latency, output length, visible content, and the risk of hitting the completion ceiling. The site’s vLLM thinking-budget guide owns the separate question of numeric reasoning sub-limits. This page only requires the batch file to declare its mode rather than inherit an unseen default.

The pinned generation config supplies temperature 1 and top-p 0.95 defaults. Our fixture explicitly uses temperature 0 and top-p 1 so the sampling policy is visible. Greedy decoding removes sampling randomness; it is not by itself a bitwise reproducibility guarantee across scheduling, kernels, hardware, or software changes.

Preflight seven failures before GPU startup

The upstream Pydantic models validate much of the request body, but an operator needs stronger whole-file rules. A comment in the pinned vLLM source says custom_id must be unique; the editorial preflight enforces it across lines. It also prevents a mixed checkpoint, streaming request, implicit thinking policy, and unsafe output ceiling from reaching expensive startup.

Deliberate mutation	Required preflight result	Why it matters
Duplicate the first row	duplicate-custom-id	Two outputs could claim one source record
Change POST to GET	method-must-be-post	The file contract is POST-oriented
Change URL to /v1/responses	unsupported-endpoint	This guide covers chat completions only
Change one body model to base GLM-5.2	model-mismatch	One engine cannot honestly represent mixed artifact coordinates
Set stream=true	streaming-not-allowed	Pinned runner source rejects streaming output
Remove template kwargs	thinking-policy-must-be-explicit	Avoid inheriting default max reasoning silently
Set completion ceiling to zero	invalid-max-completion-tokens	Reject an unusable or malformed release job

The exact fixture rejects all seven. A compact preflight can follow this shape:

preflight-batch.mjs

import fs from 'node:fs';

const path = process.argv[2];
const expectedModel = 'zai-org/GLM-5.2-FP8';
const lines = fs.readFileSync(path, 'utf8').trim().split('\n');
const rows = lines.map((line, i) => {
  try { return JSON.parse(line); }
  catch { throw new Error(`line ${i + 1}: invalid JSON`); }
});
const seen = new Set();

for (const [i, row] of rows.entries()) {
  const id = row.custom_id;
  if (typeof id !== 'string' || !id) throw new Error(`line ${i + 1}: invalid custom_id`);
  if (seen.has(id)) throw new Error(`${id}: duplicate custom_id`);
  seen.add(id);
  if (row.method !== 'POST') throw new Error(`${id}: method must be POST`);
  if (row.url !== '/v1/chat/completions') throw new Error(`${id}: unsupported endpoint`);
  if (row.body?.model !== expectedModel) throw new Error(`${id}: model mismatch`);
  if (row.body?.stream === true) throw new Error(`${id}: stream must be off`);
  const cap = row.body?.max_completion_tokens;
  if (!Number.isInteger(cap) || cap < 1 || cap > 8192) throw new Error(`${id}: invalid cap`);
  const kwargs = row.body?.chat_template_kwargs;
  const off = kwargs?.enable_thinking === false;
  const effort = ['high', 'max'].includes(kwargs?.reasoning_effort);
  if (off === effort) throw new Error(`${id}: choose exactly one thinking policy`);
}

console.log(`validated ${rows.length} unique requests`);

Extend it with your message schema, source-record lookup, privacy policy, maximum prompt bytes, allowed tools, structured-output contract, and per-task completion ceiling. The preflight should run before loading 755.7 GB of weights, not after a malformed row consumes cluster time.

Set batch and memory caps intentionally

“Offline” does not remove scheduling or memory pressure. The pinned source says all requests are submitted to the engine concurrently, and the scheduler then decides how tokens are batched. A file with 20,000 rows is not a promise that 20,000 full contexts fit simultaneously, but it is a large in-memory job and a large failure domain.

Start with three separate limits:

• File rows: how many logical jobs are lost or ambiguous if the process exits before the output file is written.
• --max-num-seqs: how many sequences the scheduler admits concurrently; this competes for KV-cache capacity.
• --max-model-len: the context ceiling exposed by this run; do not default to the model’s one-million-token maximum unless the topology was sized and tested for it.

The official recipe’s Docker example caps the model at 131,072 tokens, and its H20 example uses 16 sequences and 32,768 batched tokens. Those are evidence that the controls matter, not universal production values. Our first canary uses 131,072 and 16 as explicit, conservative starting coordinates, then reduces them if engine startup or representative prompts lack headroom.

Inspect prompt-length distribution before launch. Four 1K-token prompts and four 100K-token prompts have the same row count but radically different cache and prefill demands. Cap prompt bytes in preflight, tokenize a sample with the pinned tokenizer, and preserve p50, p95, and maximum prompt-token counts in the batch manifest. Do not claim full 1M context because the model config advertises it; capacity belongs to the exact checkpoint, engine, parallel layout, concurrency, KV dtype, and hardware combination.

Run a small pinned batch

Use a new task directory and verify that the output does not already exist. The following is a canary command, not a universal topology prescription:

Pinned vLLM offline-batch canary

export GLM52_BATCH_MODEL='zai-org/GLM-5.2-FP8'
export GLM52_BATCH_REV='ba978f7d347eaf65d22f1a86833408afdb953541'
export GLM52_BATCH_DIR='/srv/glm52-batches/job-2026-08-28-001'
export VLLM_BATCH_INVARIANT='1'

node preflight-batch.mjs "$GLM52_BATCH_DIR/requests.jsonl"
test ! -e "$GLM52_BATCH_DIR/results.jsonl"

vllm run-batch \
  -i "$GLM52_BATCH_DIR/requests.jsonl" \
  -o "$GLM52_BATCH_DIR/results.jsonl" \
  --model "$GLM52_BATCH_MODEL" \
  --revision "$GLM52_BATCH_REV" \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8_e4m3 \
  --max-model-len 131072 \
  --max-num-seqs 16 \
  --reasoning-parser glm45

Confirm those flags against vllm run-batch --help in the exact installed image before allocating the full job. The static source audit found the revision and reasoning-parser arguments in the pinned engine arguments, but it did not execute the CLI.

Keep the first correctness canary simple. Do not simultaneously enable MTP, raise context, raise concurrency, change KV dtype, add tools, and add structured output. The official recipe documents five-token MTP for GLM-5.2, but speculative decoding is another acceptance variable. Establish plain output parity and result accounting first; introduce MTP in a one-change A/B with the same input hash.

Expect concurrency, not a durable queue

The v0.23.0 source reads the entire file, builds a future for each row, awaits asyncio.gather(*response_futures), and only then calls the output writer. For a local path, that writer opens the destination with "w" and prints all responses. This creates four operational consequences:

1. A pre-existing output file can be overwritten.
2. A process or host failure before the final write can leave no durable per-row progress receipt.
3. One huge input creates one huge recovery and investigation domain.
4. Re-running the same file after an ambiguous exit can duplicate side effects if prompts or tools do anything beyond pure generation.

For pure text inference, duplication may only waste GPU time. For tool-enabled requests, duplication can be consequential. This guide’s canary has no tools. If a later batch does, require idempotency keys and a separate tool-effect ledger; do not treat custom_id as automatic idempotency enforcement.

Use chunk directories such as chunk-0001, chunk-0002, and chunk-0003. Each directory should contain the input JSONL, input hash, expected ID list, engine coordinates, output JSONL, output hash, audit report, and promotion state. Mark a chunk complete only after reconciliation. A progress bar reaching 100% is not the same as a durable release receipt.

Join every result by custom ID

The pinned source describes custom_id as the developer-provided identifier used to match outputs to inputs. Use it, even if one engine version happens to write output in input order. Positional joins are fragile across engines, chunk merges, retries, sorting, and future implementations.

Our synthetic result rows were deliberately shuffled. The audit still produced the correct mapping:

Output position	custom_id	HTTP status	Finish reason	Disposition
1	plan-0004	200	stop	Accept
2	classify-0003	400	none	Quarantine
3	extract-0001	200	stop	Accept
4	review-0002	200	length	Quarantine

The input order was extract, review, classify, plan. A line-number join would attach every output to the wrong source record. The custom_id join finds two accepted IDs—extract-0001 and plan-0004—and two quarantined IDs— classify-0003 and review-0002.

Before inspecting content, enforce set equality:

Whole-file ID gate

expected IDs - observed IDs = empty
observed IDs - expected IDs = empty
duplicate observed IDs        = empty

Any non-empty set blocks chunk promotion. Do not fill a missing row with an empty string, assume an unknown row belongs to the nearest prompt, or silently keep the first duplicate.

Quarantine errors and length truncation

A non-empty assistant string is not sufficient for release. The result wrapper can carry a non-200 response.status_code, a null body, or a non-null error. Inside a successful chat response, the first choice can end with stop, length, tool_calls, or another reason.

For the bounded text workflow here, a row is accepted only when all are true:

• custom_id belongs to the expected set and appears once;
• response.status_code equals 200;
• top-level error is null;
• response.body and choices[0] exist;
• assistant content is a non-empty string;
• finish_reason equals stop;
• prompt plus completion tokens equal total tokens.

finish_reason="length" is a failure for release purposes. The synthetic review row has content and exactly 512 completion tokens, but it stops mid- sentence at its configured ceiling. The audit quarantines it instead of presenting partial prose as a complete review. Raising the cap may be the right remediation, but only after checking prompt length, reasoning policy, expected answer reserve, and the task’s maximum safe cost.

An HTTP error also remains a separate row-level failure. Preserve its original ID, status, and error object. Retrying only failed rows can be safe for pure generation after the cause is understood; silently resubmitting the entire chunk destroys a clean one-run receipt.

Reconcile usage and file receipts

The synthetic audit sums 568 prompt tokens and 704 completion tokens for 1,272 total tokens. The arithmetic includes both accepted and quarantined rows when the response supplied usage. Excluding failed-quality rows from the bill or capacity report would understate what the engine generated.

For a real chunk, store at least:

Receipt field	Why it is required
Input and output SHA-256	Detects any later byte change
Expected, observed, missing, duplicate, and unknown ID counts	Proves whole-file accounting
Counts by HTTP status and finish reason	Separates transport, validation, truncation, and normal stop
Prompt, completion, and total tokens	Supports capacity and cost analysis
Accepted and quarantined ID lists	Makes promotion reviewable
Runtime, model, image, driver, and engine arguments	Defines the execution environment
Start, end, and audit timestamps	Establishes batch chronology

Hash bytes after the writer closes and before moving the result into a released location:

Create immutable file receipts

sha256sum "$GLM52_BATCH_DIR/requests.jsonl"
sha256sum "$GLM52_BATCH_DIR/results.jsonl"

Do not hash a normalized JSON object and call it the file hash. Preserve both if normalization is useful: one SHA-256 for exact on-disk bytes, plus a clearly labeled semantic manifest for sorted IDs and derived counts.

Make reproducibility an acceptance test

The official vLLM 0.23.0 batch-invariance example offers two controls: disable V1 multiprocessing to make scheduling deterministic, or set VLLM_BATCH_INVARIANT=1 for results intended to remain consistent across scheduling. Those are upstream generic controls, not a published GLM-5.2 FP8 quality guarantee.

Canary them on the actual workload:

1. Freeze input bytes, checkpoint revision, engine image, topology, arguments, and environment.
2. Run the same small chunk twice into two fresh output paths.
3. Join both outputs by custom_id.
4. Compare status, finish reason, token IDs when available, normalized content, and task-level acceptance—not just file hash, because request IDs and timestamps may differ.
5. Repeat after changing exactly one variable, such as batch-invariance mode or MTP.

If exact token identity is required, make that an explicit gate. If semantic equivalence is enough, define the metric and tolerance before seeing results. Temperature zero is useful, but floating-point kernels, parallel reduction, scheduler behavior, model changes, and parser changes remain separate sources of variation.

Resume with chunk manifests

There is no honest “resume from 63%” merely because the progress display reached that point before a crash. Resume from durable facts:

• If no output exists, the chunk is unfinished.
• If an output exists but its hash or ID audit is missing, the chunk is pending reconciliation.
• If the expected and observed ID sets differ, the chunk is incomplete and quarantined.
• If all IDs reconcile but some rows failed, preserve the original chunk and build a new retry chunk containing only approved IDs.
• If all rows pass, mark the original input/output pair complete and immutable.

Give the retry chunk a new job ID, path, input hash, and execution receipt while retaining parent_custom_id or another explicit lineage field in your own manifest. The runner’s file format only defines custom_id; lineage is an operator responsibility.

Never append new requests to a completed input file or overwrite its result. That would make the old hashes and counts false. New work belongs in a new chunk, even when it uses the same prompt template.

Know when a server or hosted API is better

Move away from the local file runner when you need live admission, independent client retries, per-request timeouts, queue priority, cancellation, streaming, multi-tenant authentication, autoscaling, or durable progress across node loss. Those are service or workflow requirements, not JSONL formatting details.

A hosted API may also be cheaper than provisioning a 893 GB-class FP8 topology for an occasional batch. Compare current token prices, data handling, availability, maximum batch size, supported chat_template_kwargs, output retention, and retry semantics. Do not assume a hosted provider accepts vLLM’s extra request fields or the same model revision.

Use the local runner when checkpoint control, data locality, or already-owned capacity is worth the operational work. Use a server when requests arrive over time. Use a durable workflow when the job must survive process, host, or region failure with a trustworthy state machine.

Rent only after the fixture gate

Validate a bounded offline batch on the intended accelerator topology

Rent only after the JSONL preflight, checkpoint verification, and output-audit logic pass without a model. Confirm that the exact region can supply the required accelerator count and interconnect, persistent storage exceeds the verified snapshot and runtime needs, the image is pinned by digest, and billing stops when the task-owned resources are deleted. Begin with four harmless prompts and one fresh output path.

Check current GPU options on RunPod →

Affiliate disclosure: This is an affiliate link and a referral link. GLM52.ai may receive a commission or referral credit after qualifying usage at no added charge from us. A rental does not prove GLM-5.2 availability, checkpoint integrity, vLLM compatibility, GPU count, VRAM, interconnect, storage, price, privacy, security, batch invariance, output correctness, or deletion. Verify every live property and keep your own receipts.

A four-row canary should prove startup, parser behavior, ID reconciliation, finish-reason handling, and cleanup before a large input is uploaded. A rented node does not turn static source evidence into runtime proof; preserve the actual engine log, input and output hashes, audit receipt, and resource-deletion confirmation.

Frequently asked questions

Does vLLM run-batch start an HTTP server?

No separate inference server is required for the file job. The command builds a local engine and applies OpenAI-compatible request handling to each JSONL row. Metrics can be exposed separately when enabled, but that does not turn the workflow into the complete Batch REST API.

Can one JSONL file mix GLM-5.2 and GLM-5.2-FP8?

Not in this workflow. Start one engine for one pinned model coordinate and require every body model field to match it. A mixed-model file weakens capacity planning, receipt identity, and failure diagnosis. Create separate chunks and separate output paths for different checkpoints.

Is output order guaranteed to match input order?

The audited v0.23.0 implementation gathers futures and currently writes the result list after completion, but consumers should not use position as an identity contract. The documented join field is custom_id. Sorting, merging, retrying, using another engine, or changing implementation can break a positional assumption.

Should a length-truncated row be retried automatically?

Quarantine it first. Determine whether the cap, prompt length, thinking mode, or requested answer shape caused truncation. Then create a new retry chunk with an explicit change and preserved lineage. Do not overwrite the original result or rerun every successful row.

Does VLLM_BATCH_INVARIANT guarantee identical GLM-5.2 text?

The vLLM example says the mode targets consistency across scheduling. This audit did not run GLM-5.2 or prove exact output identity on any topology. Treat the flag as a canary variable and compare actual token or task-level results on the pinned environment.

Can I use tools in the batch file?

The chat schema and GLM template can represent tools, but tools expand the failure and side-effect surface. Establish text-only accounting first. A tool batch needs a pinned tool-call parser, schema validation, effect idempotency, authorization, network boundaries, and a separate ledger. Never assume custom_id prevents a duplicated external action.

Is the synthetic receipt evidence that GLM-5.2 produced good output?

No. The four outputs are deliberately synthetic fixtures for testing the auditor. They generated zero model tokens. The evidence proves that the audit accepts two well-formed stopped rows and quarantines an error plus a truncated row; it says nothing about model quality, speed, GPU fit, or batch invariance.

Sources and method

Checked on August 28, 2026. The source audit pinned and hashed 17 public snapshots, including:

• the vLLM 0.23.0 offline batch guide and offline inference guide;
• the pinned run_batch.py implementation, engine arguments, frontend arguments, release-era model table, and batch-invariance example;
• the pinned vLLM GLM-5.2 recipe and current model-table source at audited commit 5f213ed1592903b7bc38f173d320dac1b2769303;
• the official GLM-5.2 base artifacts and GLM-5.2 FP8 artifacts;
• the mandatory Z.ai GLM-5.2 release and Zhipu AI research index.

All seven AI HOT leads in this batch were classified as weak GLM links: the generic AI notebooks, Hy4 preview, Midjourney editor, Gemini transcription, NVIDIA revenue forecast, xAI litigation report, and Claude key-management note did not create a direct GLM-5.2 reader job. They remain recorded with their AI HOT permalinks in the public receipt. Proactive first-party research supplied the topic.

One exact Google query—GLM-5.2 vLLM offline batch inference JSONL—was sent through the budgeted SerpAPI client. Request serpapi-864ccd5a53dc4d74aa77f50bec6ef68b consumed one unit and returned an HTTP error. It was evaluated failed, not retried, not paginated, and did not change the decision.

The local evidence calculator parsed the pinned source markers and public model metadata, validated four request objects, applied seven mutations, and audited four shuffled synthetic result objects. Every invariant passed. It made zero model calls, downloaded no weights, started no vLLM process, ran no third-party CLI, and used no GPU. The article therefore separates source-supported file behavior, deterministic fixture behavior, and runtime properties still requiring a real canary.

About the editorial role

Evan Brooks

AI Infrastructure Editor

Evan Brooks serves as GLM52.ai’s disclosed editorial pen name for infrastructure, deployment, and training guides. The role examines checkpoint size, memory math, provider limits, implementation evidence, and operational risk. The GLM52.ai Editorial Team produces and fact-checks every article under this byline.

The GLM52.ai Editorial Team accepts responsibility for this article. Read our editorial policy or request a correction.

An AI image tool created this portrait. It does not depict a real individual.