AI Sec Watch

Researchers discovered ASCII Smuggling, a technique using Unicode Tags Block characters (special Unicode codes that mirror ASCII but stay invisible in UI elements) to hide prompt injections (tricky instructions hidden in AI input) that large language models interpret as regular text. This attack is particularly dangerous for LLMs because they can both read these hidden messages and generate them in responses, enabling more sophisticated attacks beyond traditional methods like XSS (cross-site scripting, injecting malicious code into websites) and SSRF (server-side request forgery, tricking a server into making unauthorized requests).

A researcher discovered that Anthropic's Claude AI model is vulnerable to hidden prompt injections using Unicode Tags code points (invisible characters that can carry secret instructions in text). Like ChatGPT before it, Claude can interpret these hidden instructions and follow them, even though users cannot see them on their screen. The researcher reported the issue to Anthropic, but the ticket was closed without further details provided.

CVE-2024-0964 is a vulnerability in Gradio (an AI tool library) where an attacker can remotely read files from a server by sending a specially crafted JSON request. The flaw exists because Gradio doesn't properly limit which files users can access through its API, allowing attackers to bypass directory restrictions and read sensitive files they shouldn't be able to reach.

Google Bard gained a code interpreter feature that lets it run Python code to create charts and perform calculations. The feature works by executing code in a sandboxed environment (an isolated virtual computer), which users can trigger by asking Bard to visualize data or plot results. While exploring this sandbox, the author found it to be somewhat unreliable and less capable than similar features in other AI systems, with limited ability to run arbitrary programs.

LlamaIndex (a tool for building AI applications with custom data) versions up to 0.9.34 has a SQL injection vulnerability (a flaw where attackers can insert malicious database commands into normal text input) in its Text-to-SQL feature. This allows attackers to run harmful SQL commands by hiding them in English language requests, such as deleting database tables.

LlamaHub (a library for loading plugins) versions before 0.0.67 have a vulnerability in how they handle OpenAPI and ChatGPT plugin loaders that allows attackers to execute arbitrary code (run any code they choose on a system). The problem is that the code uses unsafe YAML parsing instead of safe_load (a secure function that prevents malicious code in configuration files).

A researcher discovered that Amazon Q for Business was vulnerable to an indirect prompt injection attack (a technique where an attacker hides malicious instructions in data that gets fed to an AI), which could trick the AI into outputting markdown tags that render as hyperlinks. This allowed attackers to steal sensitive data from victims by embedding malicious links in uploaded files. Amazon identified and fixed the vulnerability after the researcher reported it.

A researcher discovered that LLMs like ChatGPT can be tricked through prompt injection (hiding malicious instructions in input text) by using invisible Unicode characters from the Tags Unicode Block (a section of the Unicode standard containing special code points). The proof-of-concept demonstrated how invisible instructions embedded in pasted text caused ChatGPT to perform unintended actions, such as generating images with DALL-E.

NVIDIA Triton Inference Server for Linux and Windows has a vulnerability (CVE-2023-31036) that occurs when launched with the non-default --model-control explicit option, allowing attackers to use path traversal (exploiting how file paths are handled to access unintended directories) through the model load API. A successful attack could lead to code execution (running unauthorized commands), denial of service (making the system unavailable), privilege escalation (gaining higher access levels), information disclosure (exposing sensitive data), and data tampering (modifying files).