SPLASH: Contrastive Audit Pressure for Detecting and Localizing Security Hallucinations in Generated Code

Summary

SPLASH is a method for detecting security hallucinations in LLM-generated code, which pass functional tests while staying insecure. It scores candidate programs using audit pressure, a contrast between teacher-forced log-probabilities under a security review and a style review. On 63,684 programs from 12 models across 899 tasks, it picks the secure program in 74% of cases and places the vulnerable statement first in 58% of insecure programs.