What GLM-5.2 Cybersecurity Benchmarks Actually Show

Maya Chen

Model Research & Evaluation Editor

Editorial responsibility: GLM52.ai Editorial Team · Published and editorially reviewed July 26, 2026

How we research and review · Report a correction

Disclosed editorial pen name · AI-generated portrait

The useful reading

GLM-5.2 shows credible bounded cybersecurity skill, but the public evidence does not support unattended end-to-end cyber operations. AISI found it comparable to several earlier closed models on narrow tasks, yet it stalled around step 11 of a 32-step range. Irregular independently observed useful intermediate work but no completed long-horizon scenario.

Original visualization of rounded values published by UK AISI and CAISI/NIST on July 17 and July 23, 2026. The bars describe different tests and must not be combined into one score.

GLM-5.2’s cybersecurity results look contradictory only when unlike tests are treated as the same job. One report calls it the most cyber-capable open-weight model tested in June. Another gives it 24% on ExploitBench. A cyber range shows progress to step 11 of 32, while a separate lab reports zero completed long-horizon scenarios.

Those findings can all be true. They measure technical depth, exploit-ladder progress, and sustained orchestration under different harnesses and budgets. This page reconciles the evidence without running an exploit or repeating a vendor leaderboard. It also turns the published numbers into a deployment rule for authorized defensive work.

Scope and authorization

This is benchmark interpretation, not an exploitation guide. Use security models only against systems you own or are explicitly authorized to test. The calculations below run no model, target, scanner, or offensive tool.

In this analysis

1. Read the evidence as three different jobs
2. A 24% ExploitBench score is not a 24% hack rate
3. The step-11 stall measures orchestration, not zero skill
4. Narrow tasks show the other side of the model
5. Cost changes the pilot threshold, not the safety threshold
6. Do not merge incompatible benchmark rows
7. Choose a deployment boundary from the failure mode
8. Reproduce the arithmetic without running an exploit
9. Security controls to add before model access
10. Questions the scorecard cannot answer
11. Sources and method

Read the evidence as three different jobs

The most defensible conclusion comes from keeping each evaluation attached to its task:

Evidence, checked July 26	What the model had to do	Published GLM-5.2 result	What the result does not establish
AISI narrow cyber tasks	solve bounded vulnerability research, reverse engineering, web exploitation, and cryptography tasks	comparable to Opus 4.6 and GPT-5.3-Codex across the four difficulty levels	reliable completion of a long dependent chain
CAISI/NIST ExploitBench	progress through a 16-capability exploitation ladder over 41 V8 vulnerabilities	rounded score of 24%	compromise 24% of arbitrary software or networks
AISI “The Last Ones” range	sustain a 32-step simulated corporate-network attack	average progress to step 11 within a 100M-token limit	success against defended production infrastructure
Irregular’s three suites	combine atomic work, multi-stage scenarios, and open-ended research	three Atomic Task solves; no CyScenarioBench or FrontierCyber solve	zero useful intermediate analysis

The GLM-5.2 benchmark hub separates official and independent evidence across more domains. This page goes narrower: it asks whether the cyber evidence supports bounded assistance, autonomous execution, or neither.

A 24% ExploitBench score is not a 24% hack rate

ExploitBench rejects the idea that “the program crashed” and “the agent achieved arbitrary code execution” are equivalent. Its authors define 16 measurable capabilities, including coverage, crash reproduction, memory primitives, control-flow hijack, and arbitrary code execution. Deterministic oracles grade progress across 41 post-2023 V8 vulnerabilities.

The joint UK AISI / CAISI assessment reports 24% for GLM-5.2 and 32% for Kimi K3. The CAISI/NIST mirror publishes the same rounded values and methodology boundary.

The arithmetic is simple:

percentage-point gap = 32% - 24% = 8 points
relative score ratio = 32 / 24 = 1.33×

The difference is 8 percentage points. The 1.33× figure is a ratio between two rounded scores on this harness. It is not evidence that Kimi is 33% better at security in general, and it says nothing about safe deployment. The public prose does not give GLM’s count at every ladder milestone, so converting 24% into “about ten vulnerabilities solved” would be an unsupported shortcut.

The distinction matters for defenders. A model that regularly finds a crash may accelerate triage. A model that sometimes constructs a memory primitive may help an expert investigate reachability. Neither observation proves reliable arbitrary code execution or permission to act on a live target.

The step-11 stall measures orchestration, not zero skill

“The Last Ones” (TLO) is a 32-step simulated corporate network spanning four subnets and roughly 20 hosts. AISI gives agents initial access and measures progress through an intentional attack path. Within a 100M-token limit, the July 23 comparison reports these average positions:

Model group	Average step	Descriptive share of 32 steps
GLM-5.2	11	34.4%
Kimi K3	17	53.1%
leading U.S. models in the comparison	28.5	89.1%

We calculated the last column by dividing the average step by 32. It makes scale visible; it is not a probability of compromise. Steps differ in difficulty, later actions depend on earlier ones, and average progress does not reveal the distribution across runs.

The GLM result still contains a useful clue. AISI’s July 17 analysis says GLM-5.2 reached step 7 with marginally fewer tokens than any other model on average and tracked Opus 4.6 to step 11 before stalling. That pattern points to efficient early progress followed by an orchestration ceiling.

The range also omits active defenders, defensive tooling, and penalties for noisy actions. It contains a deliberate route to the objective. Reaching step 11 there must not be presented as reaching one-third of a real enterprise compromise.

Narrow tasks show the other side of the model

AISI’s narrow suite uses 70 tasks, five attempts per task, and up to 2.5M tokens per attempt. The tasks span 18 technical-non-expert, 25 apprentice, 19 practitioner, and 8 expert items. AISI found GLM-5.2 comparable to Opus 4.6 and GPT-5.3-Codex on these tasks, including across all four difficulty bands.

That result is more optimistic than TLO because the job is different. A bounded task gives the agent a shorter feedback loop and fewer dependent decisions to preserve. The maximum allocation is 350 attempts and 875M tokens, although that ceiling is not reported consumption.

Irregular’s independent GLM-5.2 assessment shows the same split through another harness. Its self-hosted model solved three Atomic Tasks and demonstrated reverse engineering, protocol manipulation, cryptographic reasoning, and exploit-development progress. It solved no CyScenarioBench challenge and no open-ended FrontierCyber target.

Irregular describes meaningful intermediate work even on failures, including source inspection, custom tooling, memory disclosure, and early-stage access. Its conclusion is narrower than “the model cannot do cyber”: the main observed gap was carrying technical progress through a long chain with enough reliability to reach the final objective.

Cost changes the pilot threshold, not the safety threshold

AISI estimated a 100M-token range run at roughly $46 for GLM-5.2 and $85 for Opus 4.5 or 4.6. The GLM estimate is $39 lower, or 45.9% less. Across tasks that both GLM-5.2 and Opus 4.6 solved with 100% reliability, AISI reported $6.12 versus $15.17 per task, making the GLM figure 59.7% lower.

Those numbers can justify a broader controlled pilot. They do not justify broader permissions. Model cost excludes the reviewer, sandbox, instrumentation, remediation, false leads, and the expected loss from an unsafe action.

AISI also states that it did not use first-party providers for the open-weight models. The figures are evaluation estimates tied to a particular setup, not current Z.ai prices or a universal self-hosting bill. Use the GLM-5.2 cost calculator for a workload-specific commercial estimate, but keep safety controls outside the token-price formula.

Do not merge incompatible benchmark rows

Five confounders prevent a single “cyber score”:

1. Outcome: ExploitBench grades ladder progress; TLO records range steps; Irregular reports suite solves and intermediate findings.
2. Budget: the tests use different token ceilings, attempt counts, stopping conditions, and agent scaffolds.
3. Safeguards: AISI says U.S. closed-weight models were tested with system-level safeguards disabled. Public products normally expose different behavior.
4. Environment: V8 bugs, a simulated network, bounded atomic tasks, and current real systems are different target classes.
5. Date and route: model versions, hosted endpoints, prompts, tools, and open-weight serving stacks can change after publication.

This is why the broad GLM-5.2 versus Kimi K3 comparison remains a separate page. The 24% versus 32% row answers one exploit-ladder question; it cannot replace pricing, coding, modality, context, or access evidence.

Choose a deployment boundary from the failure mode

Use the observed reliability profile to choose permissions:

Intended job	Evidence-led starting boundary	Stop condition
explain a suspicious code path	read-only repository and a human reviewer	model invents a path not supported by code or tests
draft a safe reproducer for an owned test fixture	disposable sandbox, no external network, synthetic data	target identity or authorization is ambiguous
rank alerts or propose triage steps	retrieved evidence attached to every recommendation	recommendation cannot be traced to a source artifact
run a multi-stage security investigation	explicit phase gates and fresh approval before each state-changing tool	the plan changes, credentials broaden, or scope drifts
autonomous production exploitation	do not grant this boundary from current public evidence	not applicable

Open weights can keep sensitive artifacts inside infrastructure you control. The official GLM-5.2 model card lists an MIT license, a 1M context claim, and a 753B model size. That is a real privacy and customization path, but it is a large infrastructure commitment. Review the local hardware requirements before treating “downloadable” as “easy to isolate.”

Prototype an isolated self-hosted evaluation

If private model hosting fits your authorization and data boundary, price a short-lived GPU environment before committing to hardware. Keep the security fixture synthetic, deny unnecessary egress, pin the serving image, and delete only the task-owned pod and storage after the evaluation.

View RunPod GPU options →

Affiliate disclosure: This is an affiliate link. GLM52.ai may receive compensation from a qualifying RunPod account at no added charge to you. That relationship does not alter the cited benchmark results, hardware caveats, or containment advice.

Reproduce the arithmetic without running an exploit

We copied only values stated in the AISI and CAISI/NIST prose into a small JSON fixture. A dependency-free script checks the expected task counts and computes the ratios:

const pointGap = 32 - 24;
const relativeRatio = 32 / 24;
const glmPathShare = (11 / 32) * 100;
const rangeCostReduction = ((85 - 46) / 85) * 100;

The script ran inside pinned node:24.4.1-alpine3.22 with no network, a read-only root filesystem, all Linux capabilities dropped, and only the evidence directory mounted read-only. It called no model and performed no security operation. The exact container was removed and the pulled image deleted after the calculation.

You can inspect the sanitized derived result JSON. Its interpretation limits travel with the numbers so a chart cannot silently turn range progress into real-world risk.

Security controls to add before model access

A strong bounded-task result increases the need for conventional controls:

• give the model read-only access by default and separate analysis from action;
• replace production credentials with scoped, expiring test identities;
• block external network access unless a reviewed task needs an allowlisted destination;
• run code in a disposable sandbox with CPU, memory, time, file, and token caps;
• validate tool arguments and require approval for writes, scans, shell execution, or credential use;
• retain prompts, tool calls, model/version IDs, artifacts, and human decisions in an audit log;
• treat generated code, commands, URLs, and findings as untrusted until independently checked;
• define an incident stop button outside the agent and test it before expanding scope.

For a hosted route, compare provider data terms, logs, and access controls in the GLM-5.2 API provider guide. For self-hosting, apply the same controls at the orchestration layer; possessing the weights does not create monitoring or authorization.

Questions the scorecard cannot answer

Is GLM-5.2 the best open-weight cybersecurity model?

AISI called it the most cyber-capable open-weight model as of its June 2026 testing. The later preliminary comparison placed Kimi K3 above it on ExploitBench and TLO. “Best” therefore needs a date, a workload, a harness, and a reproducible configuration.

Does 24% mean GLM-5.2 fully exploited about ten of 41 bugs?

No. ExploitBench is a capability ladder with 16 graded flags, and the government publication reports a rounded aggregate score. Without the underlying GLM milestone table and scoring calculation, multiplying 41 by 24% would invent a solve count.

Why can narrow-task performance look strong when range performance stalls?

Short tasks reduce planning depth and dependency management. A long range requires the agent to preserve state, choose the next action, recover from errors, and maintain a coherent objective across many tool calls. The evidence suggests GLM-5.2’s technical depth is stronger than its sustained orchestration reliability.

Can defenders still use GLM-5.2?

Yes, for authorized, bounded assistance with human verification and least-privilege tools. The results support piloting analysis, triage, and controlled test-fixture work. They do not support unattended access to production systems.

Did GLM52.ai run these offensive benchmarks?

No. We did not run a model, exploit, target, or cyber range. We verified primary reports, normalized their test boundaries, and reproduced only the published arithmetic in an isolated container.

Sources and method

Checked July 26, 2026. The topic was discovered through this AI HOT item; AI HOT was used only as a lead, not as evidence for changing claims.

• UK AISI: open-weight cyber capability gap — narrow-task setup, TLO trajectory, cost estimates, safeguards, and limitations.
• UK AISI / CAISI: preliminary Kimi K3 assessment — rounded ExploitBench scores and TLO comparison.
• CAISI/NIST mirror of the joint assessment — U.S. government publication of the same joint results.
• ExploitBench paper — 16-level capability design, 41 V8 vulnerabilities, and deterministic grading.
• Irregular’s GLM-5.2 assessment — independent bounded-task and long-horizon observations.
• Z.ai’s official GLM-5.2 model card — model identity, license, size, context, and deployment links.

The 8-point gap, 1.33× ratio, path shares, and cost reductions are our arithmetic over rounded published values. They are descriptive, not confidence intervals or predictions. AISI’s cost figures are not first-party provider quotes. Irregular did not publish enough raw GLM run data on its page for us to reproduce that evaluation, so its findings remain attributed rather than presented as our test.

About the editorial role

Maya Chen

Model Research & Evaluation Editor

Maya Chen serves as GLM52.ai’s disclosed editorial pen name for model comparisons, launch tracking, and evidence reviews. The role separates first-party specifications, vendor benchmarks, independent results, and editorial inference. The GLM52.ai Editorial Team produces and fact-checks every article under this byline.

The GLM52.ai Editorial Team accepts responsibility for this article. Read our editorial policy or request a correction.

An AI image tool created this portrait. It does not depict a real individual.