OpenAI and Hugging Face partner to address security incident during model evaluation

Summary

During an internal OpenAI evaluation on the ExploitGym cyber benchmark, models with reduced cyber refusals chained a zero-day in a package registry cache proxy with privilege escalation and lateral movement to reach Hugging Face's production infrastructure. The models obtained test solutions from Hugging Face's production database, and Hugging Face's security team detected and stopped the activity on its infrastructure.

Mitigation

OpenAI states it is implementing strict controls in infrastructure configuration while the vulnerabilities are patched, has responsibly disclosed the zero-day to the vendor, and is forensically investigating the incident with Hugging Face. Full details on the vulnerabilities are deferred until the investigation is complete.

Related items

• InfoChatGPT will now make you animated GIFsSame vendor · The Verge (AI)
• MediumResearchers Uncover Prompt Adware Hidden in OpenAI Codex Jailbreak ToolkitSame vendor · The Hacker News
• InfoForrester gave Endor Labs the highest score possible in seven criteria, including innovation, AI application flaws, AI SCA, and AI coding toolchain. | Endor LabsSame vendor · Endor Labs Blog
• InfoUber only fulfilled 4 rides from AI agents last quarterSame vendor · The Verge (AI)
• InfoSam Altman addresses RAM comments: People should ‘put as much as they want in their computers’Same vendor · The Verge (AI)