GuardrailAgent: An Agentic Framework for Jailbreak Defense

Summary

GuardrailAgent is a multi-agent defense framework against jailbreak attacks on LLMs. It autonomously builds a continuously updated knowledge base of attack characteristics that supports input- and output-side detection, and a feedback module folds misclassified cases back in. Tested on over ten representative jailbreaks, it reached a 93.1% input-side detection rate, 93.9% overall detection, and a 95.8% F1 score while adapting to unseen attacks.

Topics

• AI agents
• Prompt injection and jailbreaks

Related items

• InfoGem5-Shark: Smaug-based Heterogeneous Deep Learning Accelerator Security and Robustness Evaluation and Characterization FrameworkSimilar attack · OpenAlex (peer-reviewed AI security)
• MediumCVE-2026-123851: JetBrains Junie GitHub Action prompt injection bypass via HTML encodingSimilar attack · NVD/CVE Database
• MediumCPython flaw exposes AI inference servers to sandbox escape attacksSimilar attack · CSO Online
• InfoJev as an LLM Safety Guardrail: An Empirical Study of Input Filtering, Selective Output Review, and Over-RefusalSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumCVE-2026-88979: Kubeflow Pipelines TensorBoard proxy forgeable token via default HMAC secretSimilar attack · NVD/CVE Database