InfoResearchPreprintLLM-specific
VeriHarm-Bench: Measuring Harmful Success in LLM Coding Agents Verifiably
- Published
- Record updated
Summary
VeriHarm-Bench measures harmful success, where a coding agent completes a benign task but also causes unintended harm such as deleting unrelated files, weakening access controls, or corrupting data. The authors introduce VeriHarm, an automatic framework that minimally perturbs coding tasks and synthesizes executable harm tests alongside the original task-correctness tests. Frontier agents, including GPT-5.6-Sol and Claude-Opus-5, averaged 95% task accuracy on the curated benchmark but only 37% safe success rate.
Topics
Related items
- MediumCVE-2026-130358: AdminNeo before 5.9.1 contains a missing authentication vulnerability in the ClaudeSqlPlugin, GeminiSqlPlugin, and…Same vendor · NVD/CVE Database
- Lowllm 0.37.3Same vendor · Simon Willison's Weblog
- InfoCalifornia sues OpenAI over Hugging Face attackSame vendor · BBC Technology
- InfoAltimeter's Brad Gerstner aims to invest $5 billion in Anthropic IPOSame vendor · CNBC Technology
- InfoOpenAI inks agreements with four independent evaluators to boost company's safety workSame vendor · CNBC Technology