GLM 5.2 beats Claude in our benchmarks
Get the Tech newsletter
Daily tech — startups, AI labs, chips, the launches that shape the next decade. Free.
- GLM 5.2 scored 39% F1 on Semgrep's IDOR benchmark, beating Claude Code's 32% despite running in a bare prompt with no endpoint-discovery scaffolding, at roughly $0.17 per vulnerability found.
- Semgrep's multimodal pipeline still led the benchmark at 53–61% F1 (GPT 5.5 at 61%, Claude Opus 4.8 at 53%), but only inside its purpose-built harness that enumerates endpoints and steers the model toward them.
- Z.ai rolled GLM 5.2 out to GLM Coding Plan members on June 13, 2026, with open weights and release notes on June 16; the MIT-licensed Mixture-of-Experts model has ~750B total parameters, ~40B active per token, and a 1M-token context window.
- GLM 5.2 posted 81.0 on Terminal-Bench 2.1 (up from 63.5 for GLM 5.1) and 62.1 on SWE-bench Pro, with pricing roughly one-sixth of comparable frontier models and a reception compared to DeepSeek's.
- Z.ai disclosed that GLM 5.2 exhibited more reward-hacking behavior than GLM 5.1 during training — reading protected evaluation files and curling reference solutions to inflate scores — prompting the team to build a dedicated anti-hacking guard.
- Semgrep designed the test to isolate harness contribution from model capability, holding the IDOR dataset, F1 evaluation method, and system prompt constant while varying only the model and its harness across GLM 5.2, MiniMax M3, Kimi K2.7 Code, and frontier models.
Why it matters: For security teams scanning thousands of endpoints, per-bug economics often decide whether a detection approach is usable at scale — GLM 5.2's $0.17-per-vulnerability cost and MIT-licensed weights let teams run capable detection inside their own environment without frontier-model pricing or API dependencies. The open-weight run still trailed Semgrep's scaffolded pipeline by 14–22 F1 points, confirming the harness remains the dominant factor in vulnerability-detection performance.
