AI Sec Watch

LangChain (an AI framework for building applications with language models) version 0.0.194 contains a code injection vulnerability (CWE-94, a weakness where attackers can inject malicious code into a program) that allows attackers to execute arbitrary code through the PALChain component, specifically in the from_math_prompt and from_colored_object_prompt functions that use Python's exec command.

Anthropic patched a data exfiltration vulnerability in Claude caused by image markdown injection, a technique where attackers embed hidden instructions in image links to trick the AI into leaking sensitive information. While Microsoft fixed this vulnerability in Bing Chat and OpenAI chose not to address it in ChatGPT, Anthropic implemented a mitigation to protect Claude users from this attack.

CVE-2023-4033 is an OS command injection vulnerability (a type of attack where an attacker can run arbitrary system commands) found in MLflow, an open-source machine learning platform, in versions before 2.6.0. The vulnerability allows attackers to execute unauthorized commands on affected systems.

ChatGPT has a vulnerability where attackers can use image markdown (a way to embed images in text) to trick the system into leaking data. OpenAI recently added Custom Instructions, a feature that automatically adds instructions to every message, which attackers can abuse to install a persistent backdoor (hidden access point) that steals data through the image markdown vulnerability. This technique is similar to how attackers exploit other systems by enabling features like email forwarding after they gain initial access.

MLflow (a popular machine learning platform) versions before 2.5.0 contain a vulnerability called absolute path traversal (CWE-36, where an attacker can access files anywhere on a system by manipulating file paths). This vulnerability was identified and reported through the huntr.dev bug bounty program.

A critical vulnerability (CVE-2023-3686) was found in Bylancer QuickAI OpenAI version 3.8.1 that allows SQL injection (a technique where attackers insert malicious database commands into user input) through the 's' parameter in the /blog file's GET Parameter Handler. The attack can be launched remotely, and the vendor did not respond to early disclosure attempts.

Google Bard can be tricked through image-based prompt injection (hidden instructions placed in images that the AI then follows), as demonstrated by a researcher who embedded text in an image that caused Bard to perform unexpected actions. This vulnerability shows that AI systems that analyze images may be vulnerable to indirect prompt injection attacks (tricking an AI into ignoring its normal instructions by hiding malicious commands in user-provided content).

Auto-GPT is an experimental application that uses GPT-4 (a large language model) to demonstrate AI capabilities through a command-line interface. Before version 0.4.3, malicious websites could trick Auto-GPT's language model into outputting specially encoded text (ANSI escape sequences, which are hidden commands that control console display) that would create fake or misleading messages on the user's screen, potentially causing them to run unintended commands.

Auto-GPT versions before 0.4.3 have a path traversal vulnerability (a weakness where an attacker uses file paths like '../../../' to access files outside the intended directory) in the `execute_python_code` command that fails to validate filenames, allowing an attacker to write malicious code outside the sandbox and execute arbitrary commands on the host system. This vulnerability bypasses the Docker container (a tool that isolates applications) meant to protect the main system from untrusted code.