Skip to content
AI Sec Watch
InfoResearchPreprintLLM-specific

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

Source
Arxiv (cs.CR + cs.CL + cs.LG)(opens in a new tab)
Published
Record updated
View JSON

Summary

Researchers propose StableJudge, a lightweight defense for LLM-as-a-Judge pipelines against score-hijacking prompt injection embedded in evaluated content. A compact 0.13B Sentry localizes injected spans and sanitizes them before scoring, trained on adversarially evolving attacks, and deterministic fine-grained scores are derived from the judge's output distribution. On held-out scientific abstracts and out-of-distribution attack variants, it improves score recovery under attack while largely preserving general performance.

Mitigation

StableJudge is itself the proposed mitigation: a reusable pre-scoring 0.13B Sentry that localizes injected spans and performs targeted sanitization before evaluation, with the judge model left unchanged. Code and data will be released.

Related items