Protecting Your Customized LLM Systems From Backdoored Instructions With Metacognitive Probing

Summary

Researchers present a black-box safety auditing agent that defends customized LLM systems against backdoored system instructions, which are malicious rules hidden in system instructions and need no fine-tuning. The agent uses metacognitive probing, which leverages the model's safety-alignment mechanisms to induce it to reveal predefined triggers, and then sanitizes those triggers from user queries. Experiments across three backdoor attack algorithms and multiple state-of-the-art LLMs substantiate its effectiveness.

Mitigation

The paper proposes a black-box safety auditing agent that uses metacognitive probing to reveal triggers and sanitizes them from user queries to keep the backdoor inactive. It also covers task-specific and broad-spectrum prompt-based probing, and requiring the model to articulate its reasoning.

Related items

• HighCVE-2026-97677: IBM Langflow OSS arbitrary file write and read by authenticated flow authorsSimilar attack · NVD/CVE Database
• MediumRecent AI agent experiments reveal a risky future of machine 'teamwork' inside corporationsSimilar attack · CNBC Technology
• MediumClaude Code's “Don't Ask Again” File Exposes Passwords and Keys in 1,122 Public Copies on GitHubSimilar attack · The Hacker News
• MediumCVE-2026-118046: JS Help Desk plugin stored DOM-based cross-site scripting via jsticket_messageSimilar attack · NVD/CVE Database
• MediumARTEX abuse expands as cybercriminals target financial sectorSimilar attack · Microsoft Security Blog