AI Sec Watch

On July 18, 2025, the European Commission released draft Guidelines that explain how the EU AI Act applies to General Purpose AI models (GPAI, which are flexible AI systems that can handle many different tasks). The Guidelines define GPAI models based on a compute threshold (10²³ FLOPs, or floating point operations, a measure combining model size and training data size), require providers to document their models and report serious incidents, and impose stricter obligations on very large models trained with 10²⁵ FLOPs or more. Providers of these large models must notify the Commission within two weeks and can request reassessment of their systemic risk classification if they provide evidence the model is not actually risky.

The Code of Practice is a framework that helps developers of General Purpose AI models (large AI systems designed for many different tasks) comply with EU AI Act requirements, though following it is voluntary. New GPAI models released after August 2, 2025 must comply immediately, while older models have until August 2, 2027, with enforcement actions delayed until August 2, 2026 to give developers time to adjust.

The dedupe Python library (which uses machine learning for fuzzy matching, deduplication, and entity resolution on structured data) had a critical vulnerability in its GitHub Actions workflow that allowed attackers to trigger code execution by commenting @benchmark on pull requests, potentially exposing the GITHUB_TOKEN (a credential that grants access to modify repository contents) and leading to repository takeover.

BentoML versions 1.4.0 to 1.4.19 have an SSRF vulnerability (server-side request forgery, where an attacker tricks a server into making requests to internal or restricted addresses) in their file upload feature. An unauthenticated attacker can exploit this to force the server to download files from any URL, including internal network addresses and cloud metadata endpoints (services that store sensitive information), without any validation.

LangChain AI version 0.3.51 contains an indirect prompt injection vulnerability (a technique where attackers hide malicious instructions in data like emails to trick AI systems) in its GmailToolkit component that could allow attackers to run arbitrary code through crafted emails. However, the supplier disputes this, arguing the actual vulnerability comes from user code that doesn't follow LangChain's security guidelines rather than from LangChain itself.

This research addresses the problem of stealing attacks against healthcare APIs (application programming interfaces, which are tools that let software systems communicate with each other), where attackers try to copy or extract data from medical AI models. The authors propose a defense strategy called "adaptive teleportation" that modifies incoming queries (requests) in clever ways to fool attackers while still allowing legitimate users to get accurate results from the healthcare API.

The Month of AI Bugs 2025 is an initiative to expose security vulnerabilities in agentic AI systems (AI that can take actions on its own), particularly coding agents, through responsible disclosure and public education. The campaign will publish over 20 blog posts demonstrating exploits, including prompt injection (tricking an AI by hiding malicious instructions in its input) attacks that can allow attackers to compromise a developer's computer without permission. While some vendors have fixed reported vulnerabilities quickly, others have ignored reports for months or stopped responding, and many appear uncertain how to address novel AI security threats.

A sandbox escape vulnerability (a security flaw allowing code to break out of a restricted execution environment) was found in huggingface/smolagents version 1.14.0 that lets attackers bypass safety restrictions and achieve remote code execution (RCE, running commands on a system they don't own). The flaw is in the local_python_executor.py module, which failed to properly block Python code execution even though it had safety checks in place.

skops is a Python library for sharing scikit-learn machine learning models. Versions 0.11.0 and below have a flaw in MethodNode that allows attackers to access unexpected object fields using dot notation, potentially leading to arbitrary code execution (running any code on a system) when loading a model file.