Pharmacist: Safety Alignment Data Curation for Large Language Models Against Harmful Fine-Tuning

Summary

Pharmacist is a method that curates safety alignment data for large language models to resist harmful fine-tuning. It trains a selector to rank alignment data, up-ranking high-quality, safety-critical examples and down-ranking the rest. When combined with RepNoise and T-Vaccine, the selected subset improved defense performance by 2.60% and 3.30%, and reduced training time by 56.83% and 57.63%.

Related items

• MediumResearchers Uncover Prompt Adware Hidden in OpenAI Codex Jailbreak ToolkitSimilar attack · The Hacker News
• MediumCritical Langflow flaws enable authentication bypass and RCESimilar attack · CSO Online
• MediumFree the Model, Sell the Gateway: Prompt Adware hidden in Jailbreak BundlesSimilar attack · Check Point Research
• InfoTraceRouter++: Causal Semantic Routing Realignment via Optimal TransportSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoCrowdStrike to Acquire Dreadnode to Bring Automated AI Security Testing to the Falcon PlatformSimilar attack · CrowdStrike Blog