Skip to content
InfoResearchPreprintLLM-specific

Risk-certified prompt optimization for LLM-as-a-judge guardrails

Published
Record updated
View JSON

Summary

This paper treats LLM-as-a-judge system-prompt selection as a constrained optimization problem: maximize benign pass rate while keeping the missed-attack rate below a user-specified level α with finite-sample statistical control. It introduces two procedures, CPS, which certifies prompts from a fixed pool using simultaneous upper confidence bounds, and CRISP, which adds risk feedback to an adaptive search. Across four datasets covering prompt injection, system-prompt leakage, and sensitive-information disclosure, risk-certified selection respected the target missed-attack level on heldout tests, and CRISP helped where fixed-pool search failed to yield certifiable high-utility candidates.