AI Sec Watch

FastGPT, an AI Agent building platform, has a vulnerability in versions before 4.9.12 where the LastRoute parameter on the login page is not properly validated or cleaned of malicious code. This allows attackers to perform open redirect (sending users to attacker-controlled websites) or DOM-based XSS (injecting malicious JavaScript that runs in the user's browser).

Attackers can exploit large language models (LLMs) through "sponge attacks," which are denial of service (DoS) attacks that craft prompts designed to generate extremely long outputs, exhausting the model's resources and degrading performance. Researchers are developing methods to predict how long an LLM's response will be based on a given prompt, creating an early warning system to detect and prevent these resource-draining attacks.

New York's legislature passed the RAISE Act (Responsible AI Safety and Education Act), which would regulate frontier AI systems (the largest, most powerful AI models) if signed into law. The act requires developers of expensive AI models to publish safety plans, withhold unreasonably risky models from release, report safety incidents within 72 hours, and face penalties up to $10 million for violations.

The European Commission is recruiting up to 60 independent experts for a scientific panel to advise on general-purpose AI (GPAI, large AI models designed for many tasks) under the EU AI Act. The panel will assess systemic risks (widespread dangers affecting multiple countries or many users), classify AI models, and issue alerts when AI systems pose significant dangers to Europe. Applicants need a PhD in a relevant field, proven AI research experience, and independence from AI companies, with the deadline set for September 14th.

This article compares traditional application security (AppSec) practices with AI security, noting that familiar principles like input validation and authentication apply to both, but AI systems introduce unique risks. New attack types specific to AI, such as prompt injection (tricking an AI by hiding instructions in its input), model poisoning (tampering with training data), and membership inference attacks (determining if specific data was in training), require security engineers to develop new defensive strategies beyond traditional code-level vulnerability management.

Cursor, a code editor designed for AI-assisted programming, had a security flaw in versions before 0.51.0 where JSON files could automatically trigger web requests without user approval. An attacker could exploit this, especially after a prompt injection attack (tricking the AI with hidden instructions in its input), to make the AI agent send data to a malicious website.

CVE-2025-32711 is a command injection vulnerability (a weakness where an attacker tricks a program into running unintended commands) in Microsoft 365 Copilot that allows an unauthorized attacker to disclose information over a network. The vulnerability has a CVSS severity score of 4.0 (a moderate rating on a 0-10 scale where 10 is most severe). Microsoft has published information about this vulnerability, but the provided source does not contain specific technical details about the attack or its impact.

FastGPT is an open-source platform for building AI workflows and chatbots that uses a sandbox (an isolated container designed to safely run untrusted code). Versions before 4.9.11 had weak isolation that allowed attackers to escape the sandbox by using overly permissive syscalls (system calls, which are requests programs make to the operating system), letting them read files, modify files, and bypass security restrictions. The vulnerability is fixed in version 4.9.11 by limiting which system calls are allowed to a safer set.

Generative AI (AI systems that create new text, code, or images) is a double-edged sword in cybersecurity, helping both defenders and attackers. The case study of a fictional insurance company shows how GenAI can be used to launch cyberattacks (malicious attempts to breach computer systems) and also to defend against them, creating a difficult choice for IT leaders about whether to use AI as a defensive tool or risk falling behind attackers who already have it.