AI Sec Watch

Former OpenAI employees and experts published an open letter asking California and Delaware officials to block OpenAI's restructuring from a nonprofit organization into a for-profit company (a Public Benefit Corporation, which balances profit with public benefit). The letter argues that the restructuring would eliminate governance safeguards designed to prevent profit motives from influencing decisions about AGI (artificial general intelligence, highly autonomous systems that outperform humans at most economically valuable work), and would shift control away from a nonprofit board accountable to the public toward a board partly accountable to shareholders.

AI agents (automated systems that can take actions based on AI decisions) are easy to build with modern tools, but they face several security threats. The OWASP Gen AI Security Project held a hackathon in New York where participants intentionally created insecure agents to identify common security problems.

On April 22, 2025, the European AI Office published preliminary guidelines explaining which companies count as providers of GPAI models (general-purpose AI models, which are AI systems capable of performing many different tasks across various applications). The guidelines cover seven key topics, including defining what a GPAI model is, identifying who qualifies as a provider, handling open-source exemptions, and compliance requirements such as documentation, copyright policies, and security protections for higher-risk models.

As AI systems start connecting to real tools and databases through the Model Context Protocol (MCP, a system that lets AI models interact with external applications and data), new security risks appear that older security methods cannot fully handle. The OWASP GenAI Security Project has released research on how to secure MCP, offering defense-in-depth strategies (a layered security approach using multiple protective measures) to help developers build safer AI applications that can act independently in real time.

Version 4.9.0 is a release of the MITRE ATLAS framework, which documents attack techniques and defenses specific to AI systems. The update adds new attack methods like reverse shells (unauthorized remote access to a system), model corruption, and supply chain attacks targeting AI tools, while also updating existing security techniques and adding real-world case studies of AI-related security breaches.

Researchers created the Virology Capabilities Test (VCT), a benchmark measuring how well AI systems can solve complex virology lab problems, and found that leading AI models like OpenAI's o3 now outperform human experts in specialized virology knowledge. This is concerning because virology knowledge has dual-use potential, meaning the same capabilities that could help prevent disease could also be misused by bad actors to develop dangerous pathogens.

PyTorch (a Python package for machine learning computations) versions 2.5.1 and earlier contain a remote code execution (RCE, where an attacker can run commands on a system they don't own) vulnerability when loading models with the torch.load function set to weights_only=True. The vulnerability stems from insecure deserialization (converting data back into executable code without checking if it's safe), which allows attackers to execute arbitrary commands remotely.

Rasa Pro is a framework for building conversational AI assistants that use large language models. A vulnerability was found where voice connectors (tools that receive audio input) did not properly check user authentication even when security tokens were configured, allowing attackers to send voice data to the system without permission.

The OWASP Generative AI Security Project, an organization focused on application security, announced nine new corporate sponsors to support efforts in improving security for generative AI technologies. The sponsors, including companies like ByteDance and Trend Micro, represent increased investment and momentum in making AI systems more secure.