InfoResearchPreprintLLM-specific
VAST: Compiling Grounded Vulnerability Scenarios into Verifiable, Multi-Step Environments for Evaluating LLM Red-Team Agents
- Published
- Record updated
Summary
VAST is a declarative range compiler that builds reproducible, verifiable multi-step environments from vulnerability scenarios to evaluate LLM red-team agents. In a budget-matched prompt contrast on the TopicServer stratum, three models solved 12 to 14 of 90 instances under contract-only or generic-hint prompts, rising to 76 of 90 when the shared workflow protocol was supplied in full.