AI Sec Watch

Version 4.9.0 is a release of the MITRE ATLAS framework, which documents attack techniques and defenses specific to AI systems. The update adds new attack methods like reverse shells (unauthorized remote access to a system), model corruption, and supply chain attacks targeting AI tools, while also updating existing security techniques and adding real-world case studies of AI-related security breaches.

Researchers created the Virology Capabilities Test (VCT), a benchmark measuring how well AI systems can solve complex virology lab problems, and found that leading AI models like OpenAI's o3 now outperform human experts in specialized virology knowledge. This is concerning because virology knowledge has dual-use potential, meaning the same capabilities that could help prevent disease could also be misused by bad actors to develop dangerous pathogens.

PyTorch (a Python package for machine learning computations) versions 2.5.1 and earlier contain a remote code execution (RCE, where an attacker can run commands on a system they don't own) vulnerability when loading models with the torch.load function set to weights_only=True. The vulnerability stems from insecure deserialization (converting data back into executable code without checking if it's safe), which allows attackers to execute arbitrary commands remotely.

Rasa Pro is a framework for building conversational AI assistants that use large language models. A vulnerability was found where voice connectors (tools that receive audio input) did not properly check user authentication even when security tokens were configured, allowing attackers to send voice data to the system without permission.

The OWASP Generative AI Security Project, an organization focused on application security, announced nine new corporate sponsors to support efforts in improving security for generative AI technologies. The sponsors, including companies like ByteDance and Trend Micro, represent increased investment and momentum in making AI systems more secure.

PyTorch 2.6.0 contains a vulnerability in the torch.nn.functional.ctc_loss function (a component used for speech recognition tasks) that can cause denial of service (making the system unavailable). The vulnerability requires local access to exploit and has been publicly disclosed, though its actual existence is still uncertain.

A critical vulnerability (CVE-2025-3677) was found in lm-sys FastChat version 0.2.36 and earlier in the file apply_delta.py. The flaw involves deserialization (converting data back into code or objects, which can be dangerous if the data comes from an untrusted source) and can only be exploited by someone with local access to the affected system.

Mattermost (a team communication platform) versions 10.4.2 and earlier, 10.5.0 and earlier, and 9.11.9 and earlier don't properly block which websites their built-in AI tool can contact. This allows logged-in users to use prompt injection (tricking the AI by hiding instructions in their input) to steal data from servers that the Mattermost system can access.

The AI Safety Newsletter highlights the launch of AI Frontiers, a new publication featuring expert commentary on critical AI challenges including national security risks, resource access inequality, risk management approaches, and governance of autonomous systems (AI agents that can make decisions without human input). The newsletter presents diverse viewpoints on how society should navigate AI's wide-ranging impacts on jobs, health, and security.