AI Sec Watch

Ray, an AI compute engine, had a critical vulnerability before version 2.52.0 that allowed attackers to run code on a developer's computer (RCE, or remote code execution) through Firefox and Safari browsers. The vulnerability exploited a weak security check that only looked at the User-Agent header (a piece of information browsers send to websites) combined with DNS rebinding attacks (tricks that redirect browser requests to unexpected servers), allowing attackers to compromise developers who visited malicious websites or ads.

This research proposes a new method for protecting data privacy in deep learning (training AI models on sensitive data) by adding Gaussian noise (random values from a bell-curve distribution) to ResNets (a type of neural network with skip connections). The method aims to provide differential privacy (a mathematical guarantee that an individual's data cannot be easily identified from the model's results) while maintaining better accuracy and speed than existing privacy-protection techniques like DPSGD (differentially private stochastic gradient descent, a slower privacy-focused training method).

Fugue is a tool that lets developers run Python, Pandas, and SQL code across distributed computing systems like Spark, Dask, and Ray. Versions 0.9.2 and earlier have a remote code execution vulnerability (RCE, where attackers can run arbitrary code on a victim's machine) in the RPC server because it deserializes untrusted data using cloudpickle.loads() without checking if the data is safe first. An attacker can send malicious serialized Python objects to the server, which will execute on the victim's machine.

A WordPress plugin called 'The AI Engine for WordPress: ChatGPT, GPT Content Generator' has a vulnerability that allows attackers with Contributor-level access or higher to read any file on the server. The problem exists because the plugin doesn't properly check file paths that users provide to certain functions (the 'lqdai_update_post' AJAX endpoint and the insert_image() function), which could expose sensitive information.

Google's new Antigravity IDE inherits multiple security vulnerabilities from the Windsurf codebase it was licensed from, including remote command execution (RCE, where an attacker can run commands on a system they don't own) via indirect prompt injection (tricking an AI by hiding instructions in its input), hidden instruction execution, and data exfiltration. The IDE's default setting allows the AI to automatically execute terminal commands without human review, relying on the language model's judgment to determine if a command is safe, which researchers have successfully bypassed with working exploits.

LangChain, a framework for building AI agents and applications powered by large language models, has a template injection vulnerability (a security flaw where attackers can hide malicious code in text templates) in versions 0.3.79 and earlier and 1.0.0 through 1.0.6. Attackers can exploit this by crafting malicious template strings that access internal Python object data in ChatPromptTemplate and similar classes, particularly when an application accepts untrusted template input.

Roo Code is an AI-powered coding agent that runs inside code editors. Before version 3.26.7, a validation error allowed Roo to automatically execute commands that weren't on an allow list (a list of approved commands), which is a type of command injection vulnerability (where attackers trick a system into running unintended commands).

Langfuse, an open source platform for managing large language models, has a vulnerability in versions 2.95.0–2.95.11 and 3.17.0–3.130.x where attackers could take over user accounts if certain security settings are not configured. The attack works by tricking an authenticated user into clicking a malicious link (via CSRF, which is cross-site request forgery where an attacker tricks your browser into making unwanted requests, or phishing).

The S2B AI Assistant WordPress plugin (a tool that adds AI chatbot features to websites) has a vulnerability in versions up to 1.7.8 where it fails to check what type of files users are uploading. This allows editors and higher-level users to upload malicious files that could potentially let attackers run commands on the website server (remote code execution, or RCE).