Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming

Summary

Screwtape is a pipeline that edits real agent transcripts to insert attacks that evade action-only monitors, refining each edit until the monitor no longer flags it. An LLM verifier checks that each attack is realistic, clearly harmful, and plausibly successful. Across 252 runs on 94 transcripts, 35 attacks evaded the authors' monitors and 6 of those also evaded a held-out frontier developer monitor, confirmed through human review.

Related items

• LowA Valid Watermark Is Not a Valid Identity: Spatial-Spectral Adversarial Face Evasion with Watermark PreservationSimilar attack · Arxiv (cs.RO + cs.CV security)
• InfoTopology-aware dynamic co-evolution for robust graph learningSimilar attack · OpenAlex (peer-reviewed AI security)
• InfoDecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and VisionSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoAIRRI: Towards a Human-Readable, Machine-Resistant Reading InterfaceSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoLimits of Statistical Assurance for Embodied and Agentic AI under Adaptive AdversariesSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)