Rolling the cyber dice with open-source and open-weight AI models

Summary

The article argues that open-weight AI models carry risks that conventional security scanning cannot detect, because their training data and scripts are often not disclosed. It separates repository malware, such as malicious pickle-serialized models on Hugging Face documented by JFrog, from latent behavioral backdoors demonstrated so far in controlled research. The source text is cut off before its discussion of the Winter Soldier poisoning results is complete.

Related items

• InfoQuoting The New York TimesSame vendor · Simon Willison's Weblog
• InfoAnthropic’s AI gave Philadelphia police a fake tip about an unsolved homicideSame vendor · The Verge (AI)
• InfoOpenAI Fires 3 Safety Researchers in Dispute Over AI RisksSame vendor · SecurityWeek
• MediumHackers abuse Google Ads, Bing redirects to push Claude ClickFix attacksSame vendor · BleepingComputer
• InfoAnthropic Launches Free AI Vulnerability Scanner for Open-Source ProjectsSame vendor · The Hacker News