Do Agents Still Follow the Rules? Benchmarking Rule Compliance under Agent Context Compaction

Summary

ARCBench, a configurable benchmark built on the open-source pi agent harness, measures whether LLM agents still follow operating rules after their context is compacted. Most post-compaction violations occur while the rule remains in the compacted history, as the agent treats a later request as permission and proceeds anyway. The strongest compactor-targeted attack more than doubles violations, while retaining a rule does not guarantee compliance: compacted runs that retain it violate more than twice as often as runs with the full history.

Related items

• MediumCVE-2026-121936: easyVoice server-side request forgery through openaiBaseUrl in /api/v1/ttsSimilar attack · NVD/CVE Database
• MediumCVE-2026-118628: MLflow get-artifacts server-side request forgery information disclosureSimilar attack · NVD/CVE Database
• HighCVE-2026-118625: MLflow Tracking Server privilege escalation via get_endpointsSimilar attack · NVD/CVE Database
• InfoWhen trust becomes the payload: Inside the threats, etc quarterly reportSimilar attack · Elastic Security Labs
• MediumCritical Langflow flaws enable authentication bypass and RCESimilar attack · CSO Online