GLM-5.2 Tokenizer API: What the Contract Says

Independent research — not an official Z.ai publication.Identity and provider disclosure

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial TeamPublished and editorially reviewed August 14, 2026

How we research and review Report a correction

Disclosed editorial pen name · AI-generated portrait

Technical diagram of a GLM-5.2 token stream reaching an API contract gate with three accepted teal model slots and one empty amber slot before message, tool, and token-meter panels

Original editorial diagram of the audit boundary. The three teal rows represent models in the current published enum; the empty amber row represents an undocumented GLM-5.2 route, not an outage or failed request.

Do not rely on Z.AI’s hosted Tokenizer endpoint as a documented GLM-5.2 admission check yet. The endpoint is real, accepts full message and function definitions, and returns prompt, image, video, and total token fields. The problem is narrower: its current OpenAPI schema has not added glm-5.2 to the allowed model list.

That mismatch is easy to miss because Z.AI’s newer GLM-5.2 pages describe a 1M context window, a 128K maximum output, and the core-parameter guide recommends the Tokenizer API for input estimation. Those statements do not amend the endpoint’s model enum. A production client should not assume that “the model exists” and “this auxiliary endpoint documents the model” are the same fact.

This guide extracts the current contract, runs positive and negative fixtures against it, and shows how to budget safely while the gap remains. It does not guess how an undocumented live route behaves.

The short verdict

The current evidence supports a documentation-blocked verdict, not a service-compatible or service-incompatible verdict.

Question	Verified answer	Operational consequence
Does Z.AI publish a Tokenizer endpoint?	Yes: `POST https://api.z.ai/api/paas/v4/tokenizer`	A client can be built from an explicit OpenAPI contract
Is `glm-5.2` in that contract’s model enum?	No, as checked August 14	Do not promise GLM-5.2 support from the schema
Do newer Z.AI pages describe GLM-5.2 context and output limits?	Yes: 1M context and 128K output	Those are model facts, not an endpoint-enum amendment
Does the open model publish a GLM-5.2 tokenizer config?	Yes, at a pinned repository revision	It enables offline inspection, not hosted-token parity
Did our local GLM-5.2 fixture pass the extracted enum?	No; it failed only `MODEL_NOT_IN_DOCUMENTED_ENUM`	The local validator behaved correctly against the document
Did Z.AI reject our GLM-5.2 request?	Unknown; no authenticated request was sent	Server acceptance, counts, quota, and billing remain untested

A live service may accept more values than an older schema declares. It may also reject a value that looks plausible because the route, account, region, or product does not expose it. The safe state is therefore “requires an updated contract or an authorized bounded receipt,” not “try it repeatedly.”

Why the current pages diverge

The chronology offers a plausible explanation. The Tokenizer page’s own structured metadata reports dateModified January 28, 2026. Z.AI’s release notes date GLM-5.2 to June 16, 2026. The current enum contains models that predate GLM-5.2.

That timing is an inference about why the mismatch exists; it is not evidence about the server. Documentation can lag a backend, and generated OpenAPI can lag narrative guides. The reverse can also happen: a marketing or migration page may announce a model before every auxiliary API supports it.

Treat each layer independently:

Model capability: the GLM-5.2 guide describes the context and output architecture.
Generation contract: the Chat Completion route defines how to request output and reports actual usage for that call.
Tokenizer contract: the auxiliary endpoint defines which request bodies it documents for counting.
Account entitlement: a valid key still needs access to the selected product, model, and route.

Only the third layer is under audit here. A missing model from one endpoint does not erase the model release or invalidate the open tokenizer files.

What the Tokenizer contract explicitly declares

The official Tokenizer API reference publishes this base and path:

server: https://api.z.ai/api
method: POST
path:   /paas/v4/tokenizer
auth:   Bearer

The request requires model and messages. Messages can include system, user, and assistant roles, but the prose rule says the list must not contain only system or assistant messages. At least one user message is therefore necessary. The request may also include up to 128 function tools.

Field	Current documented constraint	Why it matters before network I/O
`model`	enum: `glm-4.6`, `glm-4.6v`, `glm-4.5`	prevents an undeclared model from being treated as guaranteed
`messages`	required; at least one item; user message required by prose	catches empty or system-only counting requests
`tools`	optional; at most 128 functions	keeps the count body aligned with the generation body
`request_id`	optional; 6–64 characters	supports traceability without an invalid identifier
`user_id`	optional; 6–128 characters; avoid sensitive data	separates an application identifier from personal data
response `usage`	prompt, image, video, and total token fields	exposes counted categories without generating output

The model enum is the gating fact. Examples using glm-4.6 are not evidence that any newer string is accepted. An OpenAPI-generated SDK may enforce the enum before it sends a request, so backend tolerance alone would not eliminate the integration problem.

The GLM-5.2 enum gap

Z.AI’s migration guide names glm-5.2, describes 1M context and 128K maximum output, and tells users to choose the model identifier. Its core parameter guide points readers to the Tokenizer API for estimates. The GLM-5.2 model guide uses max_tokens in generation examples.

None of those pages changes the Tokenizer OpenAPI block. When our parser reads that block, it gets exactly three declared values:

[
  "glm-4.6",
  "glm-4.6v",
  "glm-4.5"
]

This is a high-value distinction for long-context applications. A tokenizer preflight often decides whether to truncate history, drop retrieval chunks, or refuse an expensive request. If it silently counts with a different model, that safety decision can be wrong even when the eventual generation endpoint works.

Do not replace glm-5.2 with glm-4.6 just to satisfy the enum. That would answer a different question. Tokenization, chat-template framing, tool-schema overhead, and provider wrappers can change the count.

Reproduce the zero-call contract audit

The repository includes a small Node.js parser and five fixtures. It fetches the official Markdown/OpenAPI pages with an identifiable research user agent, pins the public tokenizer config through the exact Hugging Face model revision, extracts only contract facts, and prints a sanitized JSON receipt.

node docs/evidence/glm-5-2-tokenizer-api-2026-08-14/audit-contract.mjs

The committed machine-readable result records HTTP status, byte counts, SHA-256 hashes, the extracted schema, fixture outcomes, and limitations. It contains no API key, Authorization header, account identifier, cookie, prompt corpus, provider response, or personal data.

Non-privileged Docker access was unavailable on the test host. There was no privileged retry: we did not use sudo or elevated access because the audit needs only Node’s HTTPS, JSON, hashing, and regular-expression support. The execution made zero calls to the Tokenizer endpoint, zero generation calls, and zero grader calls.

What the five fixtures prove

The validator is deliberately narrow. It tests request shapes against facts parsed from the publication artifact; it does not emulate tokenization.

Fixture	Local result	Meaning
`glm-4.6` + one user message	pass	positive control matches the published enum and message rule
`glm-5.2` + one user message	fail only `MODEL_NOT_IN_DOCUMENTED_ENUM`	the current contract does not declare GLM-5.2
`glm-4.6` + system-only list	fail `MESSAGES_REQUIRE_USER`	prose constraint is enforced locally
`glm-4.6` + 129 functions	fail `TOOLS_EXCEED_128`	documented tool cap is enforced locally
five-character `request_id`	fail `REQUEST_ID_LENGTH_OUT_OF_RANGE`	identifier range is enforced locally

The positive control matters. Without it, a broken parser could reject every model and produce the same headline. The three other negative controls show that the validator is reading more than one convenient field.

These outcomes do not prove that the service accepts glm-4.6 for a given account, rejects glm-5.2, returns a particular token count, or prices a request in a certain way. They prove that a client generated from the current document has a justified reason to stop before guessing.

Fail closed on the published model list

If token counting controls truncation or spend, keep the provider’s documented set in a versioned policy. Unknown values should move to an explicit review path rather than falling back to another tokenizer.

const documentedTokenizerModels = new Set([
  'glm-4.6',
  'glm-4.6v',
  'glm-4.5',
]);

export function buildTokenizerRequest({ model, messages, tools = [] }) {
  if (!documentedTokenizerModels.has(model)) {
    throw new Error(`Tokenizer model is not documented: ${model}`);
  }
  if (!messages.some((message) => message.role === 'user')) {
    throw new Error('Tokenizer messages require a user role');
  }
  if (tools.length > 128) {
    throw new Error('Tokenizer contract allows at most 128 functions');
  }
  return { model, messages, tools };
}

Do not “fix” this example by manually adding glm-5.2 and leaving the policy name unchanged. Add the value only after one of these evidence events:

Z.AI updates the OpenAPI enum;
an official Z.AI SDK with a pinned version declares it;
an authorized, bounded live test establishes acceptance and your policy records that the result is ahead of the published schema.

Store the documentation hash with the review. A later hash change is a reason to re-extract the contract, not proof that support changed.

Send the request you plan to generate

A useful tokenizer preflight must represent the assembled generation request, not only the latest user sentence. Include the same system message, retained history, retrieval text, assistant content, and function definitions that the generation client will send. The OpenAPI exposes tools for this reason, even though its prose does not specify every provider-side wrapper token.

Function definitions can be large. Descriptions, property names, enums, nested schemas, and examples all consume input space when the provider serializes them. A count over messages with an empty tool list is not a safe budget for a request that later mounts 40 tools.

Keep these invariants beside the counter:

same provider product and model identifier;
same chat protocol and message order;
same system instructions and retained history;
same tool names, descriptions, and parameter schemas;
same media references and supported content types;
same truncation and prompt-template policy.

For self-hosted work, our GLM-5.2 chat-template audit shows how message and tool branches become token IDs. The GigaToken benchmark covers local tokenizer parity and CPU throughput. Neither page establishes hosted Z.AI wrapper parity.

Budget from measured tokens, not characters

The budgeting equation is simple; choosing trustworthy inputs is not:

measured prompt tokens
+ requested output cap
+ safety reserve
<= effective context limit

Use the lowest limit that applies across model architecture, provider route, account plan, and client. Z.AI labels GLM-5.2 with 1M context and 128K maximum output. The public tokenizer configuration at pinned revision b4734de4facf877f85769a911abafc5283eab3d9 stores model_max_length as 1,048,576. Those representations are close but not interchangeable promises.

For an arithmetic illustration only, using the open config’s 1,048,576-token ceiling, reserving 131,072 for output and 8,192 for uncertainty leaves 909,312 measured prompt tokens. That result is not a hosted allowance. A product route can expose less, and the provider may add framing that a local tokenizer does not reproduce.

Use the separate GLM-5.2 max-tokens budget guide to apply the output ceiling and total-context checks as two explicit gates.

Character counts are weaker still. English prose, Chinese text, source code, minified data, Unicode, special tokens, and JSON schemas produce different token-to-character ratios. Replace character heuristics with a provider receipt or a pinned local tokenizer plus a conservative margin.

Four safe paths while the contract is open

Choose a path based on what the count controls.

Need	Safe path now	Remaining limitation
Generate one bounded GLM-5.2 request	use the documented generation route and inspect its returned usage	costs a call; usage arrives after submission
Reject oversized requests before submission	use the pinned open tokenizer and exact chat template with a margin	may differ from hosted framing and policy
Generate SDKs from OpenAPI	keep GLM-5.2 blocked for this endpoint until the enum changes	cannot offer a documented hosted preflight yet
Explore whether the backend is ahead of docs	run one authorized synthetic request, once, with a unique ID and record the rendered response	account-specific success still does not update OpenAPI

If a live test is authorized, make it diagnostic rather than expensive. Send a small synthetic user message with no personal data, no media, and no tools. Click or submit once, wait for a definite response, and never interpret a 401 or 403 as a reason to cycle keys, products, or endpoints. A 200 proves acceptance for that account and time; compare the result with the generation usage before trusting it for truncation.

For a strict application, combine two gates: a conservative local preflight before the request and the actual generation usage after it. Alert when the difference exceeds a measured tolerance. That turns an undocumented assumption into observable drift.

Separate tokenizer support from model access

Z.AI’s general API and Coding Plan are distinct products and can use different paths or entitlements. A model subscription does not by itself repair an auxiliary OpenAPI enum. Conversely, a documentation gap in Tokenizer does not mean GLM-5.2 generation is unavailable.

Review the Coding Plan versus API versus self-hosting guide before choosing a route. If the goal is provider comparison rather than this contract, use the GLM-5.2 API provider guide.

Sources, method, and limits

Primary sources checked on August 14, 2026:

Z.AI Tokenizer API reference — endpoint, request schema, enum, limits, and response fields;
Z.AI GLM-5.2 migration guide — model ID, 1M context, and 128K maximum output;
Z.AI core parameter guide — relationship between input usage, context, and the Tokenizer API;
Z.AI GLM-5.2 model guide and release notes — model behavior, generation examples, and release chronology;
official GLM-5.2 model repository — pinned public tokenizer configuration;
Z.AI GLM-5.2 release page and Zhipu AI research index — fixed first-party discovery checks.

The audit parses the official artifacts, hashes their exact bytes, and evaluates five synthetic request summaries. It does not send a Bearer token, call an API endpoint, run GLM-5.2, inspect an account, measure provider tokenization, test media, test billing, or establish a service-level guarantee. Re-run it before depending on the enum because a corrected contract should change the verdict.

GLM-5.2 Tokenizer API FAQ

Does the missing enum prove Z.AI rejects GLM-5.2?

No. It proves only that the current published OpenAPI contract does not declare the model. The backend may be ahead of the document. A definitive service claim requires an authorized live receipt or updated official contract.

Can I send `glm-4.6` to count a GLM-5.2 prompt?

Not as an exact substitute. Different tokenizers and templates can produce different counts. Using a declared but different model makes the request valid while making the result answer the wrong capacity question.

Can the open GLM-5.2 tokenizer replace the hosted endpoint?

It is useful for offline estimates, prompt-template tests, and conservative gates. It cannot prove parity with hosted wrappers, media accounting, tool serialization, route limits, or billing. Pin the revision and preserve a margin.

Does a 1M context window mean I can submit 1M input tokens?

Not automatically. Context includes input plus generated output and may include provider framing. Subtract the requested output, a measured safety reserve, and any lower product or account cap before admitting the prompt.

Should I use `prompt_tokens` or `total_tokens`?

For text-only preflight, record both and verify their relationship in a real authorized response. The contract also exposes image_tokens and video_tokens, so multimodal accounting cannot be inferred from text alone. For billing, use the generation response and current provider terms rather than a prediction.

What change would close this audit?

An official enum that includes glm-5.2 would close the documentation gap. You would still need an account-specific smoke test, a comparison with actual generation usage, and monitoring for later contract drift before using the counter as a hard production truncation gate.