All tracked items across vulnerabilities, news, research, incidents, and regulatory updates.
ChatGPT's Code Interpreter (a sandbox environment that runs code) was not properly isolated between different GPTs, meaning files uploaded to one GPT were visible and could be modified by other GPTs used by the same person, creating a security risk where malicious GPTs could steal or overwrite sensitive files. OpenAI addressed this vulnerability in May 2024.
Fix: OpenAI addressed this vulnerability in May 2024. Additionally, the source recommends: 'Disable Code Interpreter in private GPTs with private knowledge files (as they will be accessible to other GPTs)' and notes that 'when creating a new GPT Code Interpreter is off by default' as one change OpenAI made. Users should avoid uploading sensitive files to Code Interpreter and use third-party GPTs with caution, especially those with Code Interpreter enabled.
Embrace The RedResearchers discovered ASCII Smuggling, a technique using Unicode Tags Block characters (special Unicode codes that mirror ASCII but stay invisible in UI elements) to hide prompt injections (tricky instructions hidden in AI input) that large language models interpret as regular text. This attack is particularly dangerous for LLMs because they can both read these hidden messages and generate them in responses, enabling more sophisticated attacks beyond traditional methods like XSS (cross-site scripting, injecting malicious code into websites) and SSRF (server-side request forgery, tricking a server into making unauthorized requests).
CVE-2024-0964 is a vulnerability in Gradio (an AI tool library) where an attacker can remotely read files from a server by sending a specially crafted JSON request. The flaw exists because Gradio doesn't properly limit which files users can access through its API, allowing attackers to bypass directory restrictions and read sensitive files they shouldn't be able to reach.
Autolab, a web-based course management system that automatically grades programming assignments, contained path traversal vulnerabilities (a type of bug where attackers can access files outside the intended directory) that allowed instructors to read arbitrary files on the system in versions before 2.12.0. This vulnerability affects the assessment functionality and has no workaround available.
LlamaIndex (a tool for building AI applications with custom data) versions up to 0.9.34 has a SQL injection vulnerability (a flaw where attackers can insert malicious database commands into normal text input) in its Text-to-SQL feature. This allows attackers to run harmful SQL commands by hiding them in English language requests, such as deleting database tables.
LlamaHub (a library for loading plugins) versions before 0.0.67 have a vulnerability in how they handle OpenAPI and ChatGPT plugin loaders that allows attackers to execute arbitrary code (run any code they choose on a system). The problem is that the code uses unsafe YAML parsing instead of safe_load (a secure function that prevents malicious code in configuration files).
A researcher discovered that Amazon Q for Business was vulnerable to an indirect prompt injection attack (a technique where an attacker hides malicious instructions in data that gets fed to an AI), which could trick the AI into outputting markdown tags that render as hyperlinks. This allowed attackers to steal sensitive data from victims by embedding malicious links in uploaded files. Amazon identified and fixed the vulnerability after the researcher reported it.
NVIDIA Triton Inference Server for Linux and Windows has a vulnerability (CVE-2023-31036) that occurs when launched with the non-default --model-control explicit option, allowing attackers to use path traversal (exploiting how file paths are handled to access unintended directories) through the model load API. A successful attack could lead to code execution (running unauthorized commands), denial of service (making the system unavailable), privilege escalation (gaining higher access levels), information disclosure (exposing sensitive data), and data tampering (modifying files).
CVE-2023-7215 is a cross-site scripting (XSS) vulnerability, a type of attack where malicious code gets injected into a webpage that a user views in their browser, found in Chanzhaoyu chatgpt-web version 2.11.1. An attacker can exploit this by manipulating the Description argument with malicious image code, and the attack can be performed remotely over the internet. The vulnerability has been publicly disclosed and may already be in use by attackers.
CVE-2023-7018 is a deserialization of untrusted data vulnerability (a flaw where an AI library unsafely processes data from untrusted sources) in the Hugging Face Transformers library before version 4.36. This weakness could potentially allow an attacker to execute malicious code through specially crafted input.
OpenAI has begun addressing a data exfiltration vulnerability (where attackers steal user data) in ChatGPT that exploits image markdown rendering during prompt injection attacks (tricking an AI by hiding instructions in its input). The company implemented a client-side validation check called 'url_safe' on the web app that blocks images from suspicious domains, though the fix is incomplete and attackers can still leak small amounts of data through workarounds.
CVE-2023-6730 is a deserialization of untrusted data vulnerability (a security flaw where a program unsafely reconstructs objects from untrusted input, potentially allowing attackers to execute malicious code) found in the Hugging Face Transformers library before version 4.36. The vulnerability has a CVSS score of 4.0, which indicates a moderate severity level (a 0-10 rating of how severe a vulnerability is).
CVE-2023-6909 is a path traversal vulnerability (a security flaw where an attacker can access files outside their intended directory using special characters like '..\'). It affects MLflow versions before 2.9.2 in the mlflow/mlflow GitHub repository. The vulnerability was discovered and reported through the huntr.dev bug bounty platform.
CVE-2023-6831 is a path traversal vulnerability (a flaw where an attacker can access files outside the intended directory by using special characters like '..\'). in MLflow versions before 2.9.2 that allows attackers to manipulate file paths and access restricted files they shouldn't be able to reach.
CVE-2023-6572 is a command injection vulnerability (a security flaw where an attacker can run unauthorized commands) in the Gradio application (a tool for building AI demos) versions prior to the main branch. The vulnerability results from improper handling of special characters that could allow attackers to execute commands on affected systems.
Fix: As a developer, a possible mitigation is to remove Unicode Tags Block text on the way in and out (meaning filter it both when users send input to your LLM and when the LLM sends responses back to users). Additionally, test your own LLM applications for this new attack vector to identify vulnerabilities.
Embrace The RedA researcher discovered that Anthropic's Claude AI model is vulnerable to hidden prompt injections using Unicode Tags code points (invisible characters that can carry secret instructions in text). Like ChatGPT before it, Claude can interpret these hidden instructions and follow them, even though users cannot see them on their screen. The researcher reported the issue to Anthropic, but the ticket was closed without further details provided.
Fix: A patch is available at https://github.com/gradio-app/gradio/commit/d76bcaaaf0734aaf49a680f94ea9d4d22a602e70, which addresses the path traversal vulnerability (CWE-22, improper limitation of pathname access).
NVD/CVE DatabaseGoogle Bard gained a code interpreter feature that lets it run Python code to create charts and perform calculations. The feature works by executing code in a sandboxed environment (an isolated virtual computer), which users can trigger by asking Bard to visualize data or plot results. While exploring this sandbox, the author found it to be somewhat unreliable and less capable than similar features in other AI systems, with limited ability to run arbitrary programs.
Fix: Upgrade to Autolab version 2.12.0 or later, which contains a patch for this vulnerability.
NVD/CVE DatabaseFix: Upgrade LlamaHub to version 0.0.67 or later, as indicated by the release notes and patch references in the source.
NVD/CVE DatabaseA researcher discovered that LLMs like ChatGPT can be tricked through prompt injection (hiding malicious instructions in input text) by using invisible Unicode characters from the Tags Unicode Block (a section of the Unicode standard containing special code points). The proof-of-concept demonstrated how invisible instructions embedded in pasted text caused ChatGPT to perform unintended actions, such as generating images with DALL-E.
A security researcher presented at the 37th Chaos Communication Congress about Large Language Models Application Security and prompt injection (tricking an AI by hiding instructions in its input). The talk covered security research findings and was made available in video and slide formats for public access.
Gradio is a Python package for building web demos of machine learning models. Versions before 4.11.0 had a file traversal vulnerability (a weakness that lets attackers read files they shouldn't access) in the `/file` route, allowing attackers to view arbitrary files on machines running publicly accessible Gradio apps if they knew the file paths.
Fix: Update Gradio to version 4.11.0 or later, where this issue has been patched.
NVD/CVE DatabaseFix: Update to Transformers version 4.36 or later. A patch is available at the GitHub commit: https://github.com/huggingface/transformers/commit/1d63b0ec361e7a38f1339385e8a5a855085532ce
NVD/CVE DatabaseFix: OpenAI implemented a mitigation by adding a client-side validation API call (url_safe endpoint) that checks whether image URLs are safe before rendering them. The validation returns {"safe":false} to prevent rendering images from malicious domains. However, the source explicitly notes this is not a complete fix and suggests OpenAI should additionally "limit the number of images that are rendered per response to just one or maybe a handful maximum" to further reduce bypass techniques. The source also notes the current iOS version 1.2023.347 (16603) does not yet have these improvements.
Embrace The RedFix: Update MLflow to version 2.9.2 or later. A patch is available at the GitHub commit referenced: https://github.com/mlflow/mlflow/commit/1da75dfcecd4d169e34809ade55748384e8af6c1
NVD/CVE DatabaseFix: Update MLflow to version 2.9.2 or later. A patch is available at https://github.com/mlflow/mlflow/commit/1da75dfcecd4d169e34809ade55748384e8af6c1.
NVD/CVE DatabaseFix: A patch is available at the GitHub commit: https://github.com/gradio-app/gradio/commit/5b5af1899dd98d63e1f9b48a93601c2db1f56520. Users should update to the main branch or apply this commit to fix the vulnerability.
NVD/CVE Database