Anthropic Red-Team Report: Zhipu's GLM-5.3 Nears Claude Mythos on Exploits — but Its Guardrails Barely Hold

On September 29, Anthropic’s frontier red team published a cybersecurity assessment of Zhipu’s GLM-5.3. The verdict cuts two ways: this Chinese open-weight model can autonomously build end-to-end exploits at a level approaching Anthropic’s own Claude Mythos Preview — but its safety guardrails are thin, with simple prompt tricks or direct weight edits letting nearly all malicious requests through.

Two Benchmarks, Small Gap

The report’s testing methodology is public.

First is ExploitBench, measuring exploit capability against known Chrome V8 vulnerabilities. Across 410 attempts, GLM-5.3 completed 50 working end-to-end exploits; Claude Mythos Preview completed 56. Barely any separation.

Second is Anthropic’s internal binary-exploitation benchmark, targeting open-source projects covered by Google OSS-Fuzz, where a perfect score requires full control-flow hijacking. GLM-5.3 achieved it on 4% of tasks; Mythos Preview on 6%.

The same comparison against last-generation models: GLM-5.2 and Claude Opus 4.6 scored zero on both tests, and Kimi K3 and DeepSeek V4.1-Flash were near zero. In other words, this isn’t the whole industry climbing together — a single open-weight model caught up to the bar.

The US NIST’s CAISI gave an independent verdict on September 17: GLM-5.3 is the “most cyber-capable open-weight model” it has evaluated, trailing the US frontier by about four months. The two institutions’ numbers corroborate each other.

Two Human-in-the-Loop Field Tests

Beyond the numbers, the report ran two field tests with humans in the loop.

One used the full GLM-5.3: a researcher, in a fully isolated sandbox, spent a day with almost no human attention and found several previously unknown vulnerabilities in a mainstream browser’s JavaScript engine. The model then chained them into a complete attack chain: a malicious webpage that escapes the browser sandbox and reads arbitrary files from the visitor’s computer. In the test, it exfiltrated an SSH private key. Anthropic says it has reported these vulnerabilities to the browser vendor.

The other used the smaller GLM-5.3-Flash: a researcher gave it public information about a freshly disclosed Chrome vulnerability (CVE-2026-11645), spent about 20 minutes preparing the task by hand, and the model ran for 8 hours on its own, producing a working attack chain. At Zhipu’s API pricing, that run cost $20.40.

All tests ran in isolated environments. The report doesn’t prove GLM-5.3 can independently attack real systems on the public internet. But it proves one thing: the manual labor between “knowing a vulnerability” and “having working exploit code” can now be automated.

Three Layers of Defense, All Bypassed

GLM-5.3 does have defenses — direct malicious instructions get refused. Zhipu said at its August 14 launch that the model has three layers of “defense in depth”: abuse detection and interception outside the API, intent checking during inference, and the model itself learning to refuse dangerous requests. The weight release was even delayed two weeks for safety evaluation and hardening.

Anthropic’s tests went around all three layers. All three leaked:

• Wrapping malicious instructions as a “red-team exercise”: the model complied in 64% of tests;
• Prefilling the model’s chain of thought with prepared reasoning: 92% passed;
• Downloading the open weights and stripping refusal behavior directly with ablation: 100% pass-through. That step took about 2,200 GPU-hours and ~$4,400. Anthropic estimates an experienced team could do it for $1,200. After stripping, the model passed everything on three public benchmarks — JailbreakBench, HarmBench, StrongREJECT — with refusal rates falling from over 90% to 2%–12%, while capability and science scores barely moved.

For comparison, the same test suite against Claude: refusal mechanisms held against forged prompts, the API offers no prefill entry point for attackers to inject tokens, and the success rate was 0%.

The harder problem: once open weights are out, they can’t be pulled back. The report notes that within days of release, developers had published unlocked versions with guardrails stripped, circulating openly online.

Same Capability, Different Fate

Five months ago, Anthropic handled its own Claude Mythos Preview completely differently: no public weights, access only through Project Glasswing for vetted cyber defenders. Per Anthropic, those defenders have used it to find and fix over 10,000 critical software vulnerabilities.

GLM-5.3’s capability is endorsed by two institutions as approaching that level — but it’s open-weight: anyone can download, modify, and redistribute it. Anthropic’s conclusion is blunt: state and non-state actors will likely use such models to cause real harm. It also called on governments: independently safety-test sufficiently capable models, and make sure defenders’ tools are no worse than attackers’.

The report isn’t entirely selfless either. Anthropic doesn’t open-source weights, and the report conveniently frames “not open-sourcing” as a safety advantage; a cheap, near-frontier Chinese open model is also its direct competitor. Still, with CAISI’s independent assessment alongside, the capability numbers largely hold up.

Zhipu’s Side

Per Zhipu’s September 16 disclosure, a month after GLM-5.3’s release, cybersecurity became its first vertical to reach scaled revenue: about 100 security companies wired GLM models into security products and workflows, covering major domestic security vendors, internet giants, research institutions, and financial firms.

The capability grew from the same base. eWeek compared Zhipu’s release materials: GLM-5.3 and GLM-5.2 share the same 743B-parameter base model — every improvement came from post-training. But on one exploit benchmark its score rose from 24.4 to 54.4; cybersecurity tasks completed within two hours jumped from 29 to 105. One version number apart in the name; a completely different risk tier.

eWeek’s advice to enterprises: when evaluating open models, don’t treat “same family” as a free pass — every major version needs its own security review.

Three Things Worth Noting

First, the window from vulnerability disclosure to working exploit code is compressing. Twenty minutes of human prep plus eight hours of model runtime can turn a freshly disclosed vulnerability into an attack chain. Security teams’ patch windows will keep shrinking.

Second, open-weight guardrails aren’t the same as closed-API guardrails. API-side interception and intent checks all stop working once weights are downloaded locally; only the model’s own alignment remains. And alignment can be stripped for a few thousand dollars.

Third, capability evaluations must be done per version. Same base, same family name — 5.2 to 5.3 can be worlds apart in safety capability. Access reviews that only check the family name will miss the real risk.

For defenders, the good news: the same capability finds vulnerabilities too — Project Glasswing defenders have already fixed 10,000+ critical bugs with it. The bad news: capability diffusion may not wait for defenders to be ready.