Jev as an LLM Safety Guardrail: An Empirical Study of Input Filtering, Selective Output Review, and Over-Refusal

Summary

This empirical study evaluates Jev, a probabilistic decision model, as a safety guardrail for a Qwen2.5-1.5B-based LLM pipeline across 970 benchmark prompts. Input filtering cut harmful-prompt answers from 527/720 (73.2%) to 17/720 (2.4%), but raised benign-prompt refusals from 3/250 (1.2%) to 27/250 (10.8%). Tiered filtering with selective output review reduced benign refusals to 10/250 (4.0%) and harmful answers to 40/720 (5.6%), with 8.1% of prompts requiring output review.

Related items

• InfoGem5-Shark: Smaug-based Heterogeneous Deep Learning Accelerator Security and Robustness Evaluation and Characterization FrameworkSimilar attack · OpenAlex (peer-reviewed AI security)
• MediumCVE-2026-123851: JetBrains Junie GitHub Action prompt injection bypass via HTML encodingSimilar attack · NVD/CVE Database
• InfoGuardrailAgent: An Agentic Framework for Jailbreak DefenseSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumCPython flaw exposes AI inference servers to sandbox escape attacksSimilar attack · CSO Online
• MediumCVE-2026-88979: Kubeflow Pipelines TensorBoard proxy forgeable token via default HMAC secretSimilar attack · NVD/CVE Database