GLM-5.2 Agent Feedback: Testing ALIGN-Style Interfaces

Independent research — not an official Z.ai publication.Identity and provider disclosure

AI Infrastructure Editor

Editorial responsibility: GLM52.ai Editorial TeamPublished and editorially reviewed August 2, 2026

How we research and review Report a correction

Disclosed editorial pen name · AI-generated portrait

Abstract GLM-5.2 agent interface with one action trapped by an opaque rejection and another crossing a structured prerequisite bridge to valid tools

An agent calls close_incident. The environment replies, “Action rejected.” What should the model do next: export a timeline, attach evidence, notify the owner, or try closing again? The model can reason about plausible workflows, but it cannot recover a prerequisite that the interface never exposed.

Now change only the observation: “Cannot close_incident because no evidence bundle is attached. Call attach_evidence next.” The recovery problem becomes a state transition rather than a guessing game. That distinction is the useful idea behind ALIGN, a research project about improving the interface between a frozen language-model agent and its environment.

This guide separates the paper’s results from a new GLM-5.2 micro-test. It then turns the measured failure into a copyable typed-error wrapper, with controls for untrusted tool output, stale prerequisites and invented next actions. The sanitized request-level data is public so you can audit every selected action rather than trusting one percentage.

Build a better GLM-5.2 recovery interface

Separate ALIGN evidence from the GLM-5.2 probe
Read what the upstream paper actually reports
Inspect the paired GLM-5.2 result
Explain the two generic-error misses
Implement a typed rejection wrapper
Keep prerequisite feedback safe
Define a complete recovery contract
Reproduce the disposable Docker probe
Decide whether the extra context pays
Extend the experiment without inflating the claim
Audit sources, method and limitations
Answer practical interface questions

Separate ALIGN evidence from the GLM-5.2 probe

There are three evidence layers, and combining them would produce a misleading claim.

Layer	What was inspected	What it can support	What it cannot support
ALIGN paper	Authors’ ALFWorld, ScienceWorld, WebShop and M3ToolEval experiments	Interface feedback can materially change outcomes on the named setups	That GLM-5.2 achieved those scores
ALIGN repository	Source at commit `4a853dd462f08eb1fbad35ab6d5b9207d860ddcf`	The released prompts and environment-specific workflow can be examined	A turnkey install or current package contract
This site’s probe	16 real `glm-5.2` API responses on eight synthetic recovery fixtures	The observed next-action difference on this exact fixture set	A population success rate, end-to-end task score or training claim

The ALIGN paper does not report a GLM-5.2 row. Its cross-model analysis names Qwen2.5-14B and Llama 3.1/3.3 variants, and its optimization workflow names other models in analyzer and optimizer roles. The correct conclusion is therefore not “ALIGN improves GLM-5.2 by the paper’s amount.” It is “ALIGN proposes an interface hypothesis that we can test separately with GLM-5.2.”

This page also differs from the GLM-5.2 benchmark hub. The hub compares broader evidence across coding, agents, context, speed and cost. Here the independent variable is one environment message, and the output is one permitted recovery action.

Read what the upstream paper actually reports

The paper was submitted on May 27, 2025. Its preliminary ALFWorld example changes generic feedback into more informative feedback and reports a Qwen2.5-7B success-rate rise from 13.4% to 31.3%. The authors then optimize environment-agent interfaces across several benchmarks.

These are the paper’s reported averages, not our measurements:

Upstream environment	Author-reported change after interface alignment
ALFWorld success rate	+45.67 percentage points
ScienceWorld score	+10.07 points
WebShop score	+6.59 points
M3ToolEval success rate	+6.39 percentage points

The authors also report average consecutive invalid actions falling from 80.46 to 28.51 on ALFWorld and from 54.70 to 27.28 on ScienceWorld. That is a useful operational measure: repeated invalid actions consume context, time and money even when the final task eventually succeeds.

The official ALIGN repository makes the approach inspectable, but the exact pinned revision is research code, not a polished installation contract. At inspection time it had three commits, its requirements.txt was blank, and environment scripts contained local model and VLLM paths. Pin the commit and audit the environment you need; do not present pip install as a reproduction that the repository does not provide.

The most transferable idea is smaller than the full optimizer: make an invalid action return the state information needed to choose a valid transition. That is what the next test isolates.

Inspect the paired GLM-5.2 result

We created eight synthetic workflows: archiving an invoice, closing an incident, publishing a report, deploying a release, refunding a payment, rotating a key, merging a dependency and updating a shared document. Every fixture listed four allowed actions, one rejected action and one hidden prerequisite.

Each fixture ran once in each condition through https://api.z.ai/api/paas/v4/chat/completions with model ID glm-5.2, thinking disabled, do_sample=false and a 96-token cap. Condition order alternated by fixture so “generic always first” could not create a fixed ordering pattern.

Observed condition	HTTP 200	Valid JSON	Allowed action	Exact expected action	Total tokens	Median sequential latency
Generic `Action rejected by environment`	8/8	8/8	8/8	6/8	1,811	1,897 ms
Prerequisite plus next valid action	8/8	8/8	8/8	8/8	1,923	1,596 ms

GLM-5.2 paired interface test summary showing six of eight generic matches and eight of eight prerequisite-rich matches across 16 API requests

The graphic is generated from the sanitized result file. The 25-point difference describes these eight fixtures only; it is not a confidence-bound estimate of production performance.

The aligned condition added 112 total tokens across eight paired fixtures, which is 14 tokens per fixture. All 16 responses remained parseable and allowlisted, so the observed difference was not caused by repairing malformed JSON or excluding failed HTTP calls.

Do not interpret the lower aligned median latency as an acceleration claim. These were sequential calls from one host without randomized repetition, provider-load controls or enough observations for a latency comparison. The latency fields are retained as receipts, not as a benchmark result.

Explain the two generic-error misses

The two mismatches are more informative than the eight aligned matches.

For close_incident, the hidden gate required attach_evidence, but generic feedback led GLM-5.2 to choose export_timeline. Exporting a timeline is a reasonable way to preserve an investigation record. Nothing in “Action rejected” told the model that the environment specifically checked for an attached evidence bundle.

For publish_report, the hidden gate required redact_pii, but the generic condition chose generate_pdf. Again, that is a plausible publication step. The environment alone knew that its external-release policy blocked the action until the PII gate ran.

Calling either choice a reasoning failure would miss the interface failure. The model selected a valid action compatible with the visible goal; the test’s expected action depended on invisible state. Once the observation exposed that state, the expected transition became identifiable.

This matters in production because “wrong” retries are often an observability problem. Before adding a longer system prompt or increasing reasoning effort, inspect whether your tool returned a typed cause, the relevant prerequisite state and a currently permitted recovery. The GLM-5.2 tool-calling guide covers the outer function loop; this pattern improves the error object inside that loop.

Implement a typed rejection wrapper

Do not let every backend invent prose errors. Map stable internal error codes to short agent-facing contracts, then verify that the suggested transition is actually in the action allowlist.

RULES = {
    "INCIDENT_EVIDENCE_MISSING": {
        "reason": "No evidence bundle is attached.",
        "required_state": "incident.evidence_bundle_id is present",
        "next_action": "attach_evidence",
        "retryable_after": "attach_evidence",
    },
    "EXTERNAL_PII_GATE_MISSING": {
        "reason": "The external-release PII gate has not run.",
        "required_state": "report.pii_review == passed",
        "next_action": "redact_pii",
        "retryable_after": "redact_pii",
    },
}

def rejection_for_agent(error_code: str, allowed_actions: set[str]) -> dict:
    rule = RULES.get(error_code)
    if not rule or rule["next_action"] not in allowed_actions:
        return {
            "status": "rejected",
            "error_code": "UNMAPPED_REJECTION",
            "reason": "The action cannot run in the current state.",
            "required_state": None,
            "next_action": None,
            "retryable": False,
        }

    return {
        "status": "rejected",
        "error_code": error_code,
        "reason": rule["reason"],
        "required_state": rule["required_state"],
        "next_action": rule["next_action"],
        "retryable": True,
        "retryable_after": rule["retryable_after"],
    }

The fail-closed branch is important. If the server cannot map a rejection or the mapped action is not available to this agent, return next_action: null. An explicit unknown is safer than a confident instruction generated from stale policy.

Feed the JSON object back as a tool observation, not as a new system message. Then require the model’s next tool call to match the same server-side allowlist. The model may propose an action; the environment remains the authority that validates and executes it.

If your agent already enforces JSON responses, keep model-output validation separate from environment-error validation. The structured-output guide shows why an accepted response-format field is not a substitute for local Schema checks.

Keep prerequisite feedback safe

More informative errors create a larger trust boundary. A web page, shell command or remote tool can return text that looks like a prerequisite while actually containing prompt injection. Never paste arbitrary downstream error text into a privileged instruction channel.

Use these controls:

Template from typed codes. The trusted orchestrator selects a reviewed template; untrusted tools do not author next_action.
Expose minimum state. Say that authorization is missing, not which token, account secret or internal policy expression failed.
Re-check at execution. A recovery suggestion can become stale between observation and action. Validate version, ownership and prerequisite again.
Constrain the transition. Suggested actions must be present in the task-scoped allowlist and must pass the same arguments and permission checks as any other tool call.
Bound repetition. Track (error_code, attempted_action, state_version). Escalate or stop after the configured repeat count instead of producing an infinite corrective loop.
Preserve provenance. Log the typed code, state version, chosen recovery and validator result. Do not log credentials or hidden chain-of-thought.

For risky operations, pair this interface with an explicit GLM-5.2 goal contract. A clear next action does not broaden the user’s objective or authorize destructive work. It only explains how the current, already authorized transition can become valid.

Define a complete recovery contract

A useful production object needs more than friendly prose. Design each field for a concrete consumer.

Field	Consumer	Required property	Failure if omitted
`error_code`	Metrics and deterministic mapping	Stable across wording changes	Failures collapse into an unsearchable text bucket
`reason`	Model and human operator	Short, factual and non-secret	Agent guesses why the tool refused
`required_state`	Planner	Describes the gate without inventing it	Planner cannot compare current and required state
`next_action`	Tool selector	Current, permitted, optional	Agent explores plausible but irrelevant actions
`retryable_after`	Loop controller	Matches a state-changing transition	Agent repeats the rejected call immediately
`state_version`	Executor	Checked again before mutation	Recovery runs against stale state
`source`	Audit trail	Identifies trusted policy mapper	Untrusted text can masquerade as authority

Not every rejection should include a next action. A rate limit might provide a verified retry time. A permission failure may require a human role change that the agent cannot perform. A safety denial may be final. Set retryable=false and explain the boundary rather than manufacturing a workaround.

Monitor invalid-action streaks as their own metric. Task success alone hides three expensive failed calls before a final correct action. Pair streak length with tokens, elapsed time and error-code frequency, then review the highest volume ambiguous interfaces. The AgentDebugX guide shows how to preserve a failed trace and distinguish heuristic diagnosis from a GLM-based judge.

Reproduce the disposable Docker probe

The archived runner pins python@sha256:37b14db89f587f9eaa890e4a442a3fe55db452b69cca1403cc730bd0fbdc8aaf. It alternates conditions, validates JSON locally and writes a sanitized result without response IDs, headers or credentials. The essential request profile is:

{
  "model": "glm-5.2",
  "messages": [
    {"role": "system", "content": "Choose one allowed next action; return JSON only."},
    {"role": "user", "content": "Task, state, allowlist, rejected action, observation"}
  ],
  "response_format": {"type": "json_object"},
  "thinking": {"type": "disabled"},
  "do_sample": false,
  "max_tokens": 96
}

Start with Docker’s normal isolated network and a bounded DNS/HTTPS preflight. Our host’s Snap Docker bridge hit its previously documented DNS failure, so this run used --network host only after that check. The container was a trusted, outbound-only Python image with no listener, published port, Docker socket, browser profile or unrelated mount. Do not copy the exception onto a machine whose normal bridge works.

The key entered through a mode-600 FIFO and standard input. It never appeared in a command argument, container configuration, output file or screenshot. After the run, the exact task container, FIFO directory and downloaded image were removed; no global Docker prune ran.

For your reproduction, keep the paired prompt, model route and expected action fixed. Change one interface field at a time, retain unsuccessful outputs, and repeat each condition enough times to estimate variability before making a production claim. Our public JSON provides the eight fixtures, selected actions, aggregate token counts and method limits for comparison.

Z.ai’s GLM-5.2 model guide documents function calling, structured output and the first-party API controls used by this kind of loop. The official release page describes the model and its context design; neither source claims our result.

Decide whether the extra context pays

On this fixture set, the additional prompt detail cost 103 prompt tokens and nine completion tokens across the aligned condition: 112 total, or 14 per paired case. It avoided two mismatches, but that arithmetic should not be turned into a universal break-even point.

In your application, calculate:

expected recovery value
  = avoided invalid calls × (tool cost + model cost + delay cost)
    - added feedback tokens
    - interface maintenance cost
    - cost of incorrect recovery suggestions

The strongest candidates are stateful tools where the environment already knows the exact missing prerequisite: approval gates, required attachments, locks, validation steps and dependency scans. The weakest candidates are open world failures where the server does not know a single correct next action.

Do not optimize prose length before correctness. A compact wrong suggestion is worse than a slightly longer next_action:null. Once mappings are correct, measure whether a stable prefix or small error catalog reduces repeated token cost; the prompt-caching guide explains the telemetry boundary for that separate optimization.

Extend the experiment without inflating the claim

Eight fixtures are enough to expose an ambiguity, not enough to certify an agent. A useful next evaluation adds depth in controlled stages:

Repeat each paired condition with randomized order and report dispersion.
Add negative controls where the environment truly cannot identify a next action; reward the model for stopping or escalating.
Test stale suggestions by changing state_version before execution.
Inject untrusted text into a downstream tool result and prove that only the typed mapper can populate privileged recovery fields.
Extend from one-step exact match to full task completion, invalid-action streak, total tokens, wall time and side effects.
Compare a short typed object with equivalent prose to learn whether schema or information content drives the improvement.
Re-run after a model, provider, prompt or tool-schema change and preserve both old and new receipts.

If you adopt the full ALIGN optimizer, reproduce its named benchmark and pin every dependency before comparing. The upstream blank requirements file and local model paths mean the repository revision alone is not a complete environment specification. Publish setup failures alongside scores instead of silently excluding them.

Audit sources, method and limitations

Primary sources were the ALIGN paper, its official repository, the GLM-5.2 model guide, the fixed Z.ai GLM-5.2 release source and the fixed Zhipu research index, which redirected to its www.zhipuai.cn HTTPS canonical during the check. The latter two exposed no official ALIGN-specific GLM-5.2 result.

The topic was discovered through the AI HOT ALIGN item, then verified against the paper and repository rather than treating the feed summary as evidence. A targeted Google result-set check found no dedicated GLM-5.2 plus ALIGN interface test, which helped preserve a distinct search intent.

Our 16 API calls used one endpoint, one date, one model ID, one sampling profile and one observation per condition. The fixtures are synthetic and their expected actions were authored by us. Aligned feedback explicitly gives the prerequisite, so its improvement measures information supplied by the interface rather than general intelligence. No credential, authorization header, provider response ID or hidden reasoning is published.

Answer practical interface questions

Was GLM-5.2 trained or evaluated with ALIGN?

We found no primary source establishing that. The ALIGN paper does not include a GLM-5.2 row, and the checked Z.ai release, model guide and Zhipu research index do not claim an ALIGN-specific GLM-5.2 result. This page applies the paper’s interface hypothesis in a separate dated probe.

Does 8/8 aligned recovery mean the pattern always works?

No. It means all eight authored fixtures returned their exact expected action once under the recorded configuration. There is no confidence interval, model comparison or end-to-end task measurement. Repeat with your own state machine and preserve failures.

Should every tool error tell the agent what to do next?

Only when a trusted environment mapper knows a current, permitted transition. For unknown, final, safety-related or human-only failures, return a stable code, a safe reason, next_action:null and the appropriate escalation boundary.

Can this test use Coding Plan instead of the metered API?

The recorded requests used Z.ai’s general pay-as-you-go endpoint. Coding Plan is a subscription for supported client paths, not a generic replacement for every custom SDK call. Keep the two access products separate when you reproduce the result.

What should I monitor after deployment?

Track invalid-action streaks, repeated error-code/action pairs, exact recovery, task completion, tool-side rejection, tokens, delay and human escalation. A drop in repeated invalid actions is more actionable than a single aggregate agent score because it points back to a specific interface contract.