AI Sec Watch

This article describes a curated database of AI literacy training programs across Europe designed to help organizations and professionals comply with Article 4 of the EU AI Act (a regulation requiring organizations to build employee understanding of AI). The programs are selected based on whether they teach what AI is, its risks and benefits, and how to use it responsibly in the workplace.

This content discusses security challenges in agentic AI (autonomous AI systems that can take actions independently), emphasizing that traditional jailbreak testing (attempts to trick AI into breaking its rules) misses real operational risks like tool misuse and data theft. The material suggests that contextual red teaming (security testing that simulates realistic attack scenarios in specific business environments) is needed to properly assess vulnerabilities in autonomous AI systems.

vLLM versions 0.6.5 through 0.8.4 have a vulnerability when using `PyNcclPipe` (a tool for peer-to-peer communication between multiple computers running the AI model) with the V0 engine. The issue is that a network communication interface called `TCPStore` was listening on all network connections instead of just the private network specified by the `--kv-ip` parameter, potentially exposing the system to unauthorized access.

Langroid, a Python framework for building AI applications, has a vulnerability in versions before 0.53.15 where the `LanceDocChatAgent` component uses pandas eval() (a function that executes Python code stored in strings) in an unsafe way, allowing attackers to run malicious commands on the host system. The vulnerability exists in the `compute_from_docs()` function, which processes user queries without proper protection.

Langroid, a Python framework for building LLM-powered applications, had a code injection vulnerability (CWE-94, a flaw where untrusted input can be executed as code) in its `TableChatAgent` component before version 0.53.15 because it used `pandas eval()` without proper safeguards. This could allow attackers to run arbitrary code if the application accepted untrusted user input.

The Trump Administration cancelled the Biden-era AI Diffusion Rule, which had regulated exports of AI chips and AI models (software trained to perform tasks) to different countries. At the same time, the administration approved major sales of advanced AI chips to the UAE and Saudi Arabia, with deals including up to 500,000 chips per year to the UAE and 18,000 advanced chips to Saudi Arabia.

ChatGPT through March 30, 2025, renders SVG documents (scalable vector graphics, a type of image format) directly in web browsers instead of displaying them as plain text, which allows attackers to inject HTML (the code that structures web pages) and potentially trick users through phishing attacks.

A vulnerability in the `preprocess_string()` function of the huggingface/transformers library (version v4.48.3) allows a ReDoS attack (regular expression denial of service, where a poorly written pattern causes the computer to do exponential amounts of work). An attacker can send specially crafted input with many newline characters that makes the function use excessive CPU, potentially crashing the application.

CVE-2025-1975 is a vulnerability in Ollama server version 0.5.11 that allows an attacker to crash the server through a Denial of Service attack by sending specially crafted requests to the /api/pull endpoint (the function that downloads AI models). The vulnerability stems from improper validation of array index access (CWE-129, which means the program doesn't properly check if it's trying to access memory locations that don't exist), which happens when a malicious user customizes manifest content and spoofs a service.