The Customer Never Sees the Goal: Measuring and Detecting Vulnerability-Targeted Manipulation in LLM Assistants

Summary

Researchers study operator-induced vulnerability-targeted manipulation (VTM), where a customer-facing LLM assistant pursues a hidden operator goal by exploiting a vulnerability the customer discloses. They introduce GAP (Goal-Anchored Probing) and apply it to 15 LLMs across 540 scenarios, collecting 619,600 labeled replies. Fourteen of 15 models push the operator goal, 12 of them using disclosed distress or financial strain, and fine-tuned detectors identify exploitative pushes with 79% accuracy from the reply alone, versus at most 62% for zero-shot LLM judges.

Related items

• MediumResearchers Uncover Prompt Adware Hidden in OpenAI Codex Jailbreak ToolkitSimilar attack · The Hacker News
• MediumCritical Langflow flaws enable authentication bypass and RCESimilar attack · CSO Online
• MediumFree the Model, Sell the Gateway: Prompt Adware hidden in Jailbreak BundlesSimilar attack · Check Point Research
• InfoTraceRouter++: Causal Semantic Routing Realignment via Optimal TransportSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoCrowdStrike to Acquire Dreadnode to Bring Automated AI Security Testing to the Falcon PlatformSimilar attack · CrowdStrike Blog