AI Sec Watch

MLflow has a vulnerability (CVE-2024-27132) where template variables are not properly sanitized, allowing XSS (cross-site scripting, where malicious code runs in a user's browser) when running an untrusted recipe in Jupyter Notebook. This can lead to client-side RCE (remote code execution, where an attacker can run commands on the user's computer) through insufficient input cleaning.

ONNX (a machine learning model format library) versions 1.15.0 and earlier have an out-of-bounds read vulnerability (accessing memory outside intended boundaries) caused by an off-by-one error in the ONNX_ASSERT and ONNX_ASSERTM functions, which handle string copying. This flaw could allow attackers to read sensitive data from memory.

ONNX (a machine learning model format) versions 1.15.0 and earlier contain a directory traversal vulnerability (a security flaw where an attacker can access files outside the intended directory) in the external_data field of tensor proto (a data structure component). This vulnerability bypasses a previous security patch, allowing attackers to potentially access files they shouldn't be able to reach.

A researcher discovered a vulnerability in Google Gemini where attackers can hide instructions in emails that trick the AI into automatically calling external tools (called Extensions) without the user's knowledge. When a user asks the AI to analyze a malicious email, the AI follows the hidden instructions and invokes the tool, which is a form of request forgery (making unauthorized requests on behalf of the user).

CVE-2023-30767 is a vulnerability in Intel's Optimization for TensorFlow before version 2.13.0 caused by improper buffer restrictions (inadequate checks on how much data can be written to a memory area). An authenticated user with local access to a system could exploit this flaw to gain higher privilege levels than they should have.

ChatGPT's Code Interpreter (a sandbox environment that runs code) was not properly isolated between different GPTs, meaning files uploaded to one GPT were visible and could be modified by other GPTs used by the same person, creating a security risk where malicious GPTs could steal or overwrite sensitive files. OpenAI addressed this vulnerability in May 2024.

Researchers discovered ASCII Smuggling, a technique using Unicode Tags Block characters (special Unicode codes that mirror ASCII but stay invisible in UI elements) to hide prompt injections (tricky instructions hidden in AI input) that large language models interpret as regular text. This attack is particularly dangerous for LLMs because they can both read these hidden messages and generate them in responses, enabling more sophisticated attacks beyond traditional methods like XSS (cross-site scripting, injecting malicious code into websites) and SSRF (server-side request forgery, tricking a server into making unauthorized requests).

A researcher discovered that Anthropic's Claude AI model is vulnerable to hidden prompt injections using Unicode Tags code points (invisible characters that can carry secret instructions in text). Like ChatGPT before it, Claude can interpret these hidden instructions and follow them, even though users cannot see them on their screen. The researcher reported the issue to Anthropic, but the ticket was closed without further details provided.

CVE-2024-0964 is a vulnerability in Gradio (an AI tool library) where an attacker can remotely read files from a server by sending a specially crafted JSON request. The flaw exists because Gradio doesn't properly limit which files users can access through its API, allowing attackers to bypass directory restrictions and read sensitive files they shouldn't be able to reach.