All tracked items across vulnerabilities, news, research, incidents, and regulatory updates.
LangChain (an AI framework for building applications with language models) version 0.0.194 contains a code injection vulnerability (CWE-94, a weakness where attackers can inject malicious code into a program) that allows attackers to execute arbitrary code through the PALChain component, specifically in the from_math_prompt and from_colored_object_prompt functions that use Python's exec command.
CVE-2023-4033 is an OS command injection vulnerability (a type of attack where an attacker can run arbitrary system commands) found in MLflow, an open-source machine learning platform, in versions before 2.6.0. The vulnerability allows attackers to execute unauthorized commands on affected systems.
MLflow (a popular machine learning platform) versions before 2.5.0 contain a vulnerability called absolute path traversal (CWE-36, where an attacker can access files anywhere on a system by manipulating file paths). This vulnerability was identified and reported through the huntr.dev bug bounty program.
A critical vulnerability (CVE-2023-3686) was found in Bylancer QuickAI OpenAI version 3.8.1 that allows SQL injection (a technique where attackers insert malicious database commands into user input) through the 's' parameter in the /blog file's GET Parameter Handler. The attack can be launched remotely, and the vendor did not respond to early disclosure attempts.
Auto-GPT is an experimental application that uses GPT-4 (a large language model) to demonstrate AI capabilities through a command-line interface. Before version 0.4.3, malicious websites could trick Auto-GPT's language model into outputting specially encoded text (ANSI escape sequences, which are hidden commands that control console display) that would create fake or misleading messages on the user's screen, potentially causing them to run unintended commands.
Auto-GPT versions before 0.4.3 have a path traversal vulnerability (a weakness where an attacker uses file paths like '../../../' to access files outside the intended directory) in the `execute_python_code` command that fails to validate filenames, allowing an attacker to write malicious code outside the sandbox and execute arbitrary commands on the host system. This vulnerability bypasses the Docker container (a tool that isolates applications) meant to protect the main system from untrusted code.
Auto-GPT versions before 0.4.3 have a security flaw where the docker-compose.yml file (a configuration file that sets up Docker containers) is mounted into the container without write protection. If an attacker tricks Auto-GPT into running malicious code through the `execute_python_file` or `execute_python_code` commands, they can overwrite this file and gain control of the host system (the main computer running Auto-GPT) when it restarts.
OpenAI removed the 'Chat with Code' plugin from its store after security researchers discovered it was vulnerable to CSRF (cross-site request forgery, where an attacker tricks a system into making unwanted actions on behalf of a user). The vulnerability allowed ChatGPT to accidentally create GitHub issues without user permission when certain plugins were enabled together.
A SQL injection vulnerability (a type of attack where an attacker inserts malicious SQL commands into input fields) exists in langchain versions before v0.0.247 in the SQLDatabaseChain component, allowing remote attackers to obtain sensitive information from databases.
CVE-2023-36188 is a vulnerability in langchain version 0.0.64 that allows a remote attacker to execute arbitrary code (running commands they shouldn't be able to run) through the PALChain parameter in Python's exec method. This is a type of injection attack (CWE-74, where an attacker tricks a system by inserting malicious code into input that gets processed as commands).
CVE-2023-36258 is a vulnerability in LangChain before version 0.0.236 that allows an attacker to execute arbitrary code (run any commands they want on a system) by exploiting the ability to use Python functions like os.system, exec, or eval (functions that can run code dynamically). This is a code injection vulnerability (CWE-94, where attackers trick a program into running unintended code).
Langchain version 0.0.171 has a vulnerability that allows arbitrary code execution (running uncontrolled commands on a system) through its load_prompt function. The vulnerability was reported in June 2023, but the provided source material does not contain detailed information about how the vulnerability works or its severity rating.
Bing Chat contained a prompt injection vulnerability (tricking an AI by hiding instructions in its input) where malicious text on websites could trick the AI into returning markdown image tags that send sensitive data to an attacker's server. When Bing Chat's client converts markdown to HTML, an attacker can embed data in the image URL, exfiltrating (stealing and sending out) information without the user knowing.
Langchain versions before v0.0.225 contained a remote code execution (RCE, where attackers can run commands on a system they don't own) vulnerability in the JiraAPIWrapper component that allowed attackers to execute arbitrary code through specially crafted input. The vulnerability was identified in the JiraAPI wrapper component of the library.
Anthropic patched a data exfiltration vulnerability in Claude caused by image markdown injection, a technique where attackers embed hidden instructions in image links to trick the AI into leaking sensitive information. While Microsoft fixed this vulnerability in Bing Chat and OpenAI chose not to address it in ChatGPT, Anthropic implemented a mitigation to protect Claude users from this attack.
Fix: Update MLflow to version 2.6.0 or later. A patch is available at the GitHub commit: https://github.com/mlflow/mlflow/commit/6dde93758d42455cb90ef324407919ed67668b9b
NVD/CVE DatabaseChatGPT has a vulnerability where attackers can use image markdown (a way to embed images in text) to trick the system into leaking data. OpenAI recently added Custom Instructions, a feature that automatically adds instructions to every message, which attackers can abuse to install a persistent backdoor (hidden access point) that steals data through the image markdown vulnerability. This technique is similar to how attackers exploit other systems by enabling features like email forwarding after they gain initial access.
Fix: Upgrade to MLflow version 2.5.0 or later. A patch is available at https://github.com/mlflow/mlflow/commit/6dde93758d42455cb90ef324407919ed67668b9b.
NVD/CVE DatabaseGoogle Bard can be tricked through image-based prompt injection (hidden instructions placed in images that the AI then follows), as demonstrated by a researcher who embedded text in an image that caused Bard to perform unexpected actions. This vulnerability shows that AI systems that analyze images may be vulnerable to indirect prompt injection attacks (tricking an AI into ignoring its normal instructions by hiding malicious commands in user-provided content).
Fix: The issue has been patched in release version 0.4.3.
NVD/CVE DatabaseFix: The issue has been patched in version 0.4.3. As a workaround, run Auto-GPT in a virtual machine or another environment in which damage to files or corruption of the program is not a critical problem.
NVD/CVE DatabaseFix: Update to Auto-GPT version 0.4.3 or later, as the issue has been patched in that version.
NVD/CVE DatabaseGoogle Docs recently added new AI features, such as automatic summaries and creative content generation, which are helpful but introduce security risks. The main concern is that using these AI features on untrusted data (information you don't know the source or reliability of) could lead to unwanted consequences, though currently attackers have limited ways to exploit these features.
Fix: Update langchain to version v0.0.247 or later.
NVD/CVE DatabaseFix: A patch is available at https://github.com/hwchase17/langchain/pull/6003
NVD/CVE DatabaseFix: Upgrade LangChain to version 0.0.236 or later.
NVD/CVE DatabaseOpenAI's plugin store contains security vulnerabilities, particularly in plugins that can act on behalf of users without adequate security review. These plugins are susceptible to prompt injection attacks (tricking an AI by hiding instructions in its input) and the Confused Deputy Problem (where an attacker can manipulate a plugin into performing harmful actions by exploiting its trust in the AI system), allowing adversaries to steal source code or cause other damage.
Fix: Update Langchain to v0.0.225 or later. A fix is available in the release v0.0.225.
NVD/CVE DatabaseA security researcher created a demonstration website that shows how indirect prompt injection (tricking an AI by hiding instructions in web content it reads) can be used to hijack ChatGPT when the browsing feature is enabled. The demo lets users explore various AI-based attacks, including data theft and manipulation of ChatGPT's responses, to raise awareness of these vulnerabilities.