DEPUTYBENCH: Evaluating Contextual Safety in Enterprise Agents

Summary

DEPUTYBENCH is an executable benchmark of 375 instances from 79 base items for testing whether enterprise agents respect organizational permissions while completing legitimate work. Across 12 models it reports 88.0% pooled safety and 79.1% paired success. In a controlled study of 27 matched tasks, restricted execution occurred more often with missing evidence (10.3%) than with explicit non-permission (3.5%), a 6.8-percentage-point difference.