AI Sec Watch

The Aiomatic WordPress plugin (versions up to 2.5.0) has a security flaw where it doesn't properly check what type of files users are uploading, allowing authenticated attackers with basic user access to upload harmful files to the server. This could potentially lead to RCE (remote code execution, where an attacker can run commands on a system they don't own), though an attacker needs to provide a Stability.AI API key value to exploit it.

A Server-Side Request Forgery (SSRF, a vulnerability where an AI system makes unwanted requests to internal or local servers on behalf of an attacker) vulnerability exists in the RequestsToolkit component of the langchain-community package version 0.0.27. The flaw allows attackers to scan ports, access local services, steal cloud credentials, and interact with local network servers because the toolkit doesn't block requests to internal addresses.

Runtime attacks on large language models are rapidly increasing, with jailbreak techniques (methods that bypass AI safety restrictions) and denial-of-service exploits (attacks that make systems unavailable) becoming more sophisticated and widely shared through open-source platforms like GitHub. The report explains that these attacks have evolved from isolated research experiments into organized toolkits accessible to threat actors, affecting production AI deployments across enterprises.

MLflow versions before 3.1.0 have a vulnerability in the gateway_proxy_handler component where it fails to properly validate the gateway_path parameter, potentially allowing SSRF (server-side request forgery, where an attacker tricks the server into making unwanted requests to internal systems). This validation gap could be exploited to access resources the attacker shouldn't be able to reach.

FastGPT, an AI Agent building platform, has a vulnerability in versions before 4.9.12 where the LastRoute parameter on the login page is not properly validated or cleaned of malicious code. This allows attackers to perform open redirect (sending users to attacker-controlled websites) or DOM-based XSS (injecting malicious JavaScript that runs in the user's browser).

Attackers can exploit large language models (LLMs) through "sponge attacks," which are denial of service (DoS) attacks that craft prompts designed to generate extremely long outputs, exhausting the model's resources and degrading performance. Researchers are developing methods to predict how long an LLM's response will be based on a given prompt, creating an early warning system to detect and prevent these resource-draining attacks.

New York's legislature passed the RAISE Act (Responsible AI Safety and Education Act), which would regulate frontier AI systems (the largest, most powerful AI models) if signed into law. The act requires developers of expensive AI models to publish safety plans, withhold unreasonably risky models from release, report safety incidents within 72 hours, and face penalties up to $10 million for violations.

The European Commission is recruiting up to 60 independent experts for a scientific panel to advise on general-purpose AI (GPAI, large AI models designed for many tasks) under the EU AI Act. The panel will assess systemic risks (widespread dangers affecting multiple countries or many users), classify AI models, and issue alerts when AI systems pose significant dangers to Europe. Applicants need a PhD in a relevant field, proven AI research experience, and independence from AI companies, with the deadline set for September 14th.

This article compares traditional application security (AppSec) practices with AI security, noting that familiar principles like input validation and authentication apply to both, but AI systems introduce unique risks. New attack types specific to AI, such as prompt injection (tricking an AI by hiding instructions in its input), model poisoning (tampering with training data), and membership inference attacks (determining if specific data was in training), require security engineers to develop new defensive strategies beyond traditional code-level vulnerability management.