AI Sec Watch

A researcher discovered data exfiltration vulnerabilities (security flaws that allow unauthorized data to leak out of a system) in several popular AI chatbots including Bing Chat, ChatGPT, and Claude, and responsibly disclosed them to the companies. Microsoft, Anthropic, and a plugin vendor fixed their vulnerabilities, but OpenAI decided not to fix an image markdown injection issue (a vulnerability where hidden code in image formatting can trick the AI into revealing data).

LangChain version 0.0.171 has a vulnerability (CVE-2023-36281) that allows a remote attacker to execute arbitrary code (run commands they shouldn't be able to run) by sending a specially crafted JSON file to the load_prompt function. The vulnerability relates to improper control of code generation, which means the application doesn't properly validate or sanitize (clean) the input before using it to create executable code.

Weaviate v.1.20.0 contains a vulnerability (CVE-2023-38976) in the handleUnbatchedGraphQLRequest function that allows remote attackers to cause a denial of service (making a service unavailable by overwhelming it with requests). The vulnerability has a CVSS score of 4.0 (a moderate severity rating).

CVE-2023-39659 is a vulnerability in langchain (an AI library) version 0.0.232 and earlier that allows a remote attacker to execute arbitrary code (run commands they choose) by sending a specially crafted script to the PythonAstREPLTool._run component. The vulnerability is caused by improper neutralization of special elements in output (a type of injection attack where untrusted input is not properly filtered before being processed).

CVE-2023-38896 is a vulnerability in langchain v.0.0.194 and earlier versions that allows a remote attacker to execute arbitrary code (run commands on a system they don't control) through the from_math_prompt and from_colored_object_prompt functions. This is an injection attack (CWE-74), where the software fails to properly filter special characters or commands that could be misused by downstream components.

LangChain version 0.0.231 has a vulnerability (CVE-2023-38860) where a remote attacker can execute arbitrary code by manipulating the prompt parameter, which is a type of code injection (CWE-94, where an attacker tricks the system into running malicious code by hiding it in input data).

CVE-2023-27506 is a vulnerability in Intel Optimization for Tensorflow software before version 2.12 involving improper buffer restrictions (a memory safety flaw where a program doesn't properly check that it stays within allocated memory). An authenticated user with local access to a system could potentially use this flaw to escalate their privileges, gaining higher-level access than they should have.

LangChain (an AI framework for building applications with language models) version 0.0.194 contains a code injection vulnerability (CWE-94, a weakness where attackers can inject malicious code into a program) that allows attackers to execute arbitrary code through the PALChain component, specifically in the from_math_prompt and from_colored_object_prompt functions that use Python's exec command.

Anthropic patched a data exfiltration vulnerability in Claude caused by image markdown injection, a technique where attackers embed hidden instructions in image links to trick the AI into leaking sensitive information. While Microsoft fixed this vulnerability in Bing Chat and OpenAI chose not to address it in ChatGPT, Anthropic implemented a mitigation to protect Claude users from this attack.