Skip to content
InfoResearchPeer-reviewedLLM-specific

CAST: A Compiler-Based Framework for Systematically Testing LLM Compositional Safety

Published
Record updated
View JSON

Summary

Researchers introduce compositional safety, the property that an LLM stays safe under multi-step workflows that decompose harmful intents, not only isolated malicious prompts. They present CAST, a compiler-inspired testing framework that uses an intermediate representation, CAIR, to generate and reassemble sub-tasks for evaluating malicious-code safety. On four LLMs across three testbeds, CAST exposed severe safety violations in strongly aligned models, with up to a 365% increase in successful test cases over baseline strategies.