The Model Knows Better: Defending Medical Multi-Agent Systems Against Prompt Injection

Summary

This research studies prompt injection against medical multi-agent systems, where LLM agents discuss a clinical case. Across two systems, two model families and three benchmarks, a single injected passage succeeds in up to 61% of cases, and collaboration does not consistently protect against it. The authors propose a training-free defense that separates patient findings from clinical judgments and checks each judgment against the findings alone, lowering attack success on MedQA to 4-7% while keeping accuracy on clean cases.

Mitigation

The authors propose a training-free defense that verifies the clinical judgments behind the system's decision: it separates the patient's findings from clinical judgments (diagnoses, recommendations and instructions) and checks each judgment against the findings alone, so the decision rests only on supported judgments.

Topics

• Prompt injection and jailbreaks

Related items

• MediumCVE-2026-95183: DeepSeek Harness context-compaction prompt injection via external contentSimilar attack · NVD/CVE Database
• InfoMobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency InjectionSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoIgnored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using AgentsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoSAFE-Mem: Secure Agentic Memory with Fine-Grained Authority and Governance-Aware RetrievalSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoIdentify, Classify, Act: Per-Instruction Controlled Compliance for LLMs over External DataSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)