CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat

Summary

CyberArena is a benchmark that pits two AI agents against each other in live, symmetric attack-and-defense matches inside isolated sandboxes, with services that change over time. Across 466 matches involving 12 frontier LLM agents and 40 challenges, it separates model capabilities that single-agent static benchmarks fail to distinguish. The authors report that defenders often patch one flaw while missing the next, and that stronger agents plant backdoors for later rounds.

Topics

• AI agents