A.S.E 2.0: Understanding How Repository Context Impacts the Security of LLM-Generated Code

Summary

Researchers built CASE-Bench, a benchmark of 145 repository-level tasks drawn from real CVEs across 51 projects, to test how repository context affects the security of LLM-generated code. Across 13 context construction methods and six LLMs, functionality-oriented context raised the functional pass rate by up to 8.2 pp but the secure pass rate by at most 3.7 pp. Withholding a safeguard lowered the secure pass rate by up to 8.1 pp, and coding agents observed the relevant safeguard in only 27-36% of trajectories.