InfoResearchPreprintLLM-specific
Risk-certified prompt optimization for LLM-as-a-judge guardrails
- Published
- Record updated
Summary
This paper treats LLM-as-a-judge system-prompt selection as a constrained optimization problem: maximize benign pass rate while keeping the missed-attack rate below a user-specified level α with finite-sample statistical control. It introduces two procedures, CPS, which certifies prompts from a fixed pool using simultaneous upper confidence bounds, and CRISP, which adds risk feedback to an adaptive search. Across four datasets covering prompt injection, system-prompt leakage, and sensitive-information disclosure, risk-certified selection respected the target missed-attack level on heldout tests, and CRISP helped where fixed-pool search failed to yield certifiable high-utility candidates.
Related items
- MediumCVE-2026-130358: AdminNeo before 5.9.1 contains a missing authentication vulnerability in the ClaudeSqlPlugin, GeminiSqlPlugin, and…Similar attack · NVD/CVE Database
- HighGHSA-8vf2-hxg5-crqh: Kubeshark: MCP `start_kubeshark` argument injection via `pod_regex` → image redirect → cluster takeoverSimilar attack · GitHub Advisory Database
- MediumCVE-2026-126681: AWS Labs Timestream for InfluxDB MCP Server exposes API tokenSimilar attack · NVD/CVE Database
- HighCVE-2026-126681 - Improper restriction of intended endpoints in AWS Labs Timestream for InfluxDB MCP ServerSimilar attack · AWS Security Bulletins
- MediumAmazon Uncovers AI Agent Campaign Targeting LiteLLM, Dify, and Other Apps for API KeysSimilar attack · The Hacker News