Safety Mid-Training: Internalizing LLM Safety as a Foundational Capability

Summary

Researchers propose safety mid-training, a stage between pre-training and post-training that jointly develops safety and general reasoning by treating safety as a learnable capability shared with general tasks. The base model generates structured documents and reasoning traces from safety and utility prompts for this stage. Relative to Qwen3-1.7B, 4B, and 8B base models, it reduces average attack success rate (ASR) by 72% to 81% while improving general capability, and the safety gains persist through subsequent SFT and GRPO.

Related items

• InfoConcerns in Benchmarking Jailbreaks: An Empirical Study of Attack Success RatesSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoBeyond Attack Success Rate: Cost-Aware Metric for Evaluating Iterative LLM JailbreaksSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoBudgeted Attack Scheduling for LLM Red-Teaming: The Roles of Static Priors, Semantic Guidance, and Online FeedbackSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoPeer Pressure: Execution-Verified Co-Evolution of Attacker and Defender Populations for Code SecuritySimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoThe Missing Compliance Class: LLM Graders That Collapse Jailbreak Success, and a Free CheckSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)