AI Sec Watch

AI regulatory sandboxes are controlled testing environments where companies can develop and test AI systems with guidance from regulators before releasing them to the public, as required by the EU AI Act (EU's new rules for artificial intelligence). These sandboxes help companies understand what regulations they must follow, protect them from fines if they follow official guidance, and make it easier for small startups to enter the market. Each EU Member State must create at least one sandbox by August 2, 2026, though different countries are taking different approaches to organizing them.

CVE-2025-46567 is a critical vulnerability in LLaMA-Factory (a tool for fine-tuning large language models) that exists before version 1.0.0. The vulnerability is in the `llamafy_baichuan2.py` script, which unsafely loads user-supplied files using `torch.load()` (a function that deserializes, or reconstructs, Python objects from saved data), allowing attackers to execute arbitrary commands by crafting a malicious file.

vLLM (a system for running large language models efficiently) versions 0.8.0 through 0.8.4 have a critical performance bug in how it processes multimodal input (text, images, audio). The bug uses an inefficient algorithm (quadratic time complexity, meaning it slows down exponentially as input size grows) when replacing placeholder tokens (special markers like <|audio_|> that get expanded into repeated tokens), which allows attackers to crash or freeze the system by sending specially crafted malicious inputs.

vLLM (a system for running AI models efficiently) versions 0.6.5 through 0.8.4 have a critical vulnerability when using mooncake integration. Attackers can execute arbitrary code remotely because the system uses pickle (an unsafe method for converting data into a format that can be transmitted) over unencrypted ZeroMQ sockets (communication channels) that listen to all network connections, making them easily accessible from the internet.

vLLM versions 0.5.2 through 0.8.4 have a security vulnerability in multi-node deployments where a ZeroMQ socket (a tool for sending messages between different computers) is left open to all network interfaces. An attacker with network access can connect to this socket to see internal vLLM data or deliberately slow down the system by connecting repeatedly without reading the data, causing a denial of service (making the system unavailable or very slow).

A ReDoS vulnerability (regular expression denial of service, where specially crafted text causes a regex to consume excessive CPU by repeatedly backtracking) was found in the huggingface/transformers library version 4.48.1, specifically in the GPT-NeoX-Japanese model's tokenizer. An attacker could exploit this by sending malicious input that causes the application to hang or crash due to high CPU usage.

Former OpenAI employees and experts published an open letter asking California and Delaware officials to block OpenAI's restructuring from a nonprofit organization into a for-profit company (a Public Benefit Corporation, which balances profit with public benefit). The letter argues that the restructuring would eliminate governance safeguards designed to prevent profit motives from influencing decisions about AGI (artificial general intelligence, highly autonomous systems that outperform humans at most economically valuable work), and would shift control away from a nonprofit board accountable to the public toward a board partly accountable to shareholders.

AI agents (automated systems that can take actions based on AI decisions) are easy to build with modern tools, but they face several security threats. The OWASP Gen AI Security Project held a hackathon in New York where participants intentionally created insecure agents to identify common security problems.

On April 22, 2025, the European AI Office published preliminary guidelines explaining which companies count as providers of GPAI models (general-purpose AI models, which are AI systems capable of performing many different tasks across various applications). The guidelines cover seven key topics, including defining what a GPAI model is, identifying who qualifies as a provider, handling open-source exemptions, and compliance requirements such as documentation, copyright policies, and security protections for higher-risk models.