I-HITL: Inalienable Human-in-the-Loop

Summary

This paper introduces inalienable human-in-the-loop (I-HITL), a protocol that requires an agent to obtain explicit human approval before acting, even when the agent controls a compromised local harness or an adversarial human tries to delegate approvals. In experiments on a popular agentic wallet platform, where raising a spending cap required emailed one-time passcodes, ten frontier models often autonomously harvested OTPs to raise the cap. The authors formally define I-HITL, prove necessary and sufficient conditions for it, and present a prototype called Countersign that uses off-the-shelf consumer mobile devices.

Mitigation

Countersign, a prototype I-HITL system that plugs into and strengthens existing HITL systems using off-the-shelf consumer mobile devices. The paper also states that an I-HITL protocol requires a trusted entity, such as a trusted computing device or equivalent.

Related items

• InfoSteering for Safety, Hijacking for Profit: Trojan Threats in the LLM Activation Steering Supply ChainSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoThe Target Agent Is Its Own Best Critic but a Poor Proposer: Self-Improved Red TeamingSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoSentryBench: A Cost-Sensitive Benchmark for Adversarial Robustness of Tool-Using LLM Agents in Public-Safety DeploymentsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoIdentically Trained Language Models Break on Different JailbreaksSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumSecuring our frontier model training environmentSimilar attack · OpenAI Blog