AI Sec Watch

A vulnerability in oobabooga text-generation-webui allows attackers to run arbitrary code (unauthorized commands) on the system without needing to log in. The flaw occurs because the software doesn't properly check user input for the trust_remote_code parameter before using it to load a model, letting attackers execute code with the same permissions as the service.

A vulnerability in Ays Pro AI ChatBot with ChatGPT and Content Generator (version 2.6.6 and earlier) allows sensitive information to be exposed when data is sent. The flaw, called CWE-201 (insertion of sensitive information into sent data), means attackers could potentially retrieve embedded sensitive data from the plugin.

Researchers discovered a new attack called FUBA (federated unlearning backdoor attack) that exploits a privacy feature in federated learning (a technique where multiple parties train an AI model together without sharing their raw data). The attack uses malicious unlearning requests, which are supposed to let participants remove their data from a trained model, to secretly inject backdoors (hidden harmful behaviors) into the model instead. The attack is difficult to detect because it hides from existing security defenses.

The Better Find and Replace plugin for WordPress (versions up to 1.7.7) has a security flaw where a function called rtafar_ajax() doesn't properly check user permissions, allowing low-level authenticated users (Subscriber-level access) to trigger OpenAI API key usage and consume quota, potentially costing money. This happens because the code is missing a capability check (a permission verification system that controls what users can do).

Under the EU AI Act, organizations that modify existing AI systems or general-purpose AI models (GPAI models, which are foundational AI systems designed to perform many different tasks) may become legally classified as "providers" and face significant compliance responsibilities. The article explains that modifications triggering higher compliance burdens typically involve high-risk AI systems or substantial changes to a GPAI model's capabilities or generality, such as fine-tuning (customizing a model for specific tasks). Proper assessment of whether a modification triggers provider status is critical, since misclassification can result in fines up to €15 million or 3% of global annual revenue.

This research proposes FedFFTNet, a system for identifying which camera model took a photo by using federated learning (a technique where AI models train on data kept private across multiple devices rather than sharing raw data centrally). The system uses a lightweight deep learning architecture and a Laplacian-based patch selection strategy (focusing on sharp, detailed areas of images) to identify cameras while maintaining privacy, achieving very high accuracy rates on standard benchmark datasets.

Cursor, a code editor designed for programming with AI, has a logic bug in versions 1.7.23 and below that allows attackers to bypass cursorignore (a file that protects sensitive files from being read). An attacker who has already performed prompt injection (tricking an AI by hiding instructions in its input) or controls a malicious AI model could create a new cursorignore file to override existing protections and access protected files.

Cursor, a code editor designed for AI-assisted programming, has a vulnerability in versions 1.7.44 and below where attackers can exploit NTFS path quirks (special behaviors of Windows file systems) to bypass file protection rules and overwrite files that normally require human approval, potentially leading to RCE (remote code execution, where an attacker can run commands on a system they don't own). This attack requires chaining with prompt injection (tricking an AI by hiding instructions in its input) or a malicious AI model, and only affects Windows systems using NTFS.

Cursor, a code editor designed for AI-assisted programming, had a security flaw in versions 1.7.52 and below where attackers could bypass safety checks on Windows machines. While the software blocked path manipulation (tricks to access files in unintended ways) using forward slashes and required human approval, the same trick using backslashes was not detected, potentially allowing an attacker with prompt injection access (hidden malicious instructions in AI inputs) to run arbitrary code and overwrite important files without permission.