CVE-2024-6960: The H2O machine learning platform uses "Iced" classes as the primary means of moving Java Objects around the cluster. Th

Summary: CVE-2024-6960 is a vulnerability in the H2O machine learning platform where the Iced format (a system for moving Java objects across a computer cluster) allows deserialization of any Java class without restrictions. An attacker can create a malicious model using Java gadgets (pre-built code snippets that can be chained together for attacks) that executes arbitrary code when imported into H2O.


CVE-2024-35199: TorchServe is a flexible and easy-to-use tool for serving and scaling PyTorch models in production. In affected versions

Summary: TorchServe (a tool for running PyTorch machine learning models in production) has a security flaw where two communication ports, 7070 and 7071, are exposed to all network interfaces instead of being restricted to localhost (the local machine only). This means anyone on a network could potentially access these ports. The vulnerability has been fixed and is available in TorchServe version 0.11.0.

Solution: Upgrade to TorchServe release 0.11.0, which includes the fix for this vulnerability. The fix was implemented in pull request #3083.


CVE-2024-35198: TorchServe is a flexible and easy-to-use tool for serving and scaling PyTorch models in production. TorchServe 's check

Summary: TorchServe (a tool for running machine learning models in production) has a security flaw where its allowed_urls check (a restriction on which websites models can be downloaded from) can be bypassed using special characters like ".." in the URL. Once a model file is downloaded through this bypass, it can be used again without the security check, effectively removing the protection.

Solution: The issue has been fixed by validating the URL without characters such as ".." before downloading (see PR #3082). TorchServe release 0.11.0 includes the fix. Users are advised to upgrade.

PyTorch
TorchServe
Amazon SageMaker
Amazon EKS


CVE-2024-21513: Versions of the package langchain-experimental from 0.0.15 and before 0.0.21 are vulnerable to Arbitrary Code Execution

Summary: Versions 0.0.15 through 0.0.20 of langchain-experimental contain a vulnerability where the code uses 'eval' (a function that runs Python code from text) on database values, allowing attackers to execute arbitrary code if they can control the input prompt and the server uses VectorSQLDatabaseChain (a component that connects language models to SQL databases). An attacker with low privileges could exploit this to break out of the application and access files or make unauthorized network connections.

Solution: Update langchain-experimental to version 0.0.21 or later.


Sorry, ChatGPT Is Under Maintenance: Persistent Denial of Service through Prompt Injection and Memory Attacks

Summary: Attackers can use prompt injection (tricking an AI by hiding malicious instructions in its input) to create fake memories in ChatGPT's memory tool, causing the AI to refuse all future responses with a maintenance message that persists across chat sessions. This creates a denial of service attack (making a service unavailable to users) that lasts until the user manually fixes it.

Solution: Users can recover by opening the memory tool, locating and removing suspicious memories created by the attacker. Additionally, users can entirely disable the memory feature to prevent this type of attack.


CVE-2024-25639: Khoj is an application that creates personal AI agents. The Khoj Obsidian, Desktop and Web clients inadequately sanitize

Summary: Khoj, an application that creates personal AI agents, has a vulnerability in its Obsidian, Desktop, and Web clients where user inputs and AI responses are not properly cleaned (sanitized). This allows attackers to inject malicious code through prompt injection (tricking the AI by hiding instructions in its input) via untrusted documents, which can trigger XSS (cross-site scripting, where malicious code runs in a user's browser when they view a webpage).

Solution: This vulnerability is fixed in version 1.13.0. Users should update to this version or later.


CVE-2024-40594: The OpenAI ChatGPT app before 2024-07-05 for macOS opts out of the sandbox, and stores conversations in cleartext in a l

Summary: The OpenAI ChatGPT app for macOS before July 5, 2024 had two security problems: it disabled the sandbox (a security boundary that limits what an app can access) and stored conversations in cleartext (unencrypted plain text) in a location that other apps could read. This meant user conversations were exposed to other programs on the same computer.

Solution: N/A -- no mitigation discussed in source.


An Introduction to the Code of Practice for General-Purpose AI

Summary: The EU AI Act Code of Practice is a voluntary set of guidelines published in July 2025 to help general-purpose AI (GPAI, large AI models used across many applications) model providers comply with new EU AI regulations during the gap period before formal European standards take effect in 2027 or later. The Code, developed by the EU AI Office and many stakeholders, covers three areas: Transparency and Copyright (for all GPAI providers) and Safety and Security (for providers of GPAI models with systemic risk, meaning those that could cause widespread harm). Though not legally binding, the Commission and EU AI Board confirmed the Code adequately demonstrates compliance with the AI Act's requirements.


CVE-2024-39236: Gradio v4.36.1 was discovered to contain a code injection vulnerability via the component /gradio/component_meta.py. Thi

Summary: Gradio v4.36.1 contains a code injection vulnerability (CWE-94, improper control of code generation) in the /gradio/component_meta.py file that can be triggered by crafted input. The vulnerability supplier disputes the report, arguing it describes a user attacking their own system rather than a genuine security flaw.


CVE-2024-37146: Flowise is a drag & drop user interface to build a customized large language model flow. In version 1.4.3 of Flowise, a

Summary: Flowise version 1.4.3 has a reflected cross-site scripting vulnerability (XSS, a type of attack where malicious code is injected into a webpage) in its `/api/v1/credentials/id` endpoint that allows attackers to inject harmful JavaScript into user sessions, potentially stealing information or redirecting users to malicious websites. The vulnerability is especially dangerous because it can be exploited without authentication in the default configuration and can be combined with other attacks to read files from the Flowise server.


CVE-2024-37145: Flowise is a drag & drop user interface to build a customized large language model flow. In version 1.4.3 of Flowise, a

Summary: Flowise version 1.4.3 has a reflected cross-site scripting vulnerability (XSS, where an attacker injects malicious code into web pages shown to users) in its `/api/v1/chatflows-streaming/id` endpoint. If using default settings without authentication, an attacker can craft a malicious URL that runs JavaScript in a user's browser, potentially stealing information, showing fake popups, or redirecting users to other websites.


CVE-2024-36423: Flowise is a drag & drop user interface to build a customized large language model flow. In version 1.4.3 of Flowise, a

Summary: Flowise version 1.4.3 has a reflected cross-site scripting vulnerability (XSS, a type of attack where malicious code is injected into a webpage) in its `/api/v1/public-chatflows/id` endpoint. An attacker can craft a malicious URL that injects JavaScript code into a user's session, potentially stealing information, showing fake popups, or redirecting users to other websites. This vulnerability is especially dangerous because the vulnerability exists in an unauthenticated endpoint (one that doesn't require a login) and can potentially be combined with other attacks to read files from the server.


CVE-2024-36422: Flowise is a drag & drop user interface to build a customized large language model flow. In version 1.4.3 of Flowise, a

Summary: Flowise version 1.4.3 contains a reflected cross-site scripting vulnerability (XSS, a type of attack where malicious code is injected into a webpage to compromise user sessions) in its chatflow endpoint that allows attackers to steal information or redirect users to other sites if the default unauthenticated configuration is used. The vulnerability occurs because when a chatflow ID is not found, the invalid ID is displayed in the error page without proper protection, letting attackers inject arbitrary JavaScript code. This XSS flaw can potentially be combined with path injection attacks (exploiting how the system handles file paths) to read files from the Flowise server.


CVE-2024-36421: Flowise is a drag & drop user interface to build a customized large language model flow. In version 1.4.3 of Flowise, A

Summary: Flowise version 1.4.3 has a CORS misconfiguration (a security setting that controls which websites can access the application), which allows any website to connect to it and steal user information. Attackers could potentially combine this flaw with another vulnerability to read files directly from the Flowise server without needing to log in.


CVE-2024-36420: Flowise is a drag & drop user interface to build a customized large language model flow. In version 1.4.3 of Flowise, th

Summary: Flowise version 1.4.3 has a vulnerability in its `/api/v1/openai-assistants-file` endpoint that allows arbitrary file read attacks (reading files on a system without permission) because the `fileName` parameter is not properly sanitized (cleaned of malicious input). This is caused by improper input validation, which is a common security weakness in software.


CVE-2024-38514: NextChat is a cross-platform ChatGPT/Gemini UI. There is a Server-Side Request Forgery (SSRF) vulnerability due to a lac

Summary: NextChat, a user interface for ChatGPT and Gemini, has a Server-Side Request Forgery vulnerability (SSRF, a flaw that lets attackers trick the server into making requests to unintended destinations) in its WebDav API endpoint because the `endpoint` parameter is not validated. An attacker could use this to make unauthorized HTTPS requests from the vulnerable server or inject malicious JavaScript code into users' browsers.

Solution: This vulnerability has been patched in version 2.12.4. Users should update to this version or later.


CVE-2024-5826: In the latest version of vanna-ai/vanna, the `vanna.ask` function is vulnerable to remote code execution due to prompt i

Summary: CVE-2024-5826 is a remote code execution vulnerability in the vanna-ai/vanna library's `vanna.ask` function, caused by prompt injection (tricking an AI by hiding instructions in its input) without code sandboxing. An attacker can manipulate the code executed by the `exec` function to gain full control of the app's backend server.


CVE-2024-4839: A Cross-Site Request Forgery (CSRF) vulnerability exists in the 'Servers Configurations' function of the parisneo/lollms

Summary: A CSRF vulnerability (cross-site request forgery, where an attacker tricks a user's browser into making unwanted requests on their behalf) exists in the 'Servers Configurations' function of parisneo/lollms-webui versions 9.6 and later, affecting services like XTTS and vLLM that lack CSRF protection. Attackers can exploit this to deceive users into installing unwanted packages without their knowledge or consent.


CVE-2024-4940: An open redirect vulnerability exists in the gradio-app/gradio, affecting the latest version. The vulnerability allows a

Summary: Gradio (a popular framework for building AI interfaces) has a vulnerability called an open redirect, which means attackers can trick the application into sending users to fake websites by exploiting improper URL validation. This can be used for phishing attacks (tricking people into revealing passwords), XSS (cross-site scripting, where attackers inject malicious code into web pages), and other exploits.


CVE-2024-37902: DeepJavaLibrary(DJL) is an Engine-Agnostic Deep Learning Framework in Java. DJL versions 0.1.0 through 0.27.0 do not pre

Summary: DeepJavaLibrary (DJL), a framework for building deep learning applications in Java, has a path traversal vulnerability (CWE-22, a flaw where an attacker can access files outside intended directories) in versions 0.1.0 through 0.27.0. This flaw allows attackers to overwrite system files by inserting archived files from absolute paths into the system.

Solution: Upgrade to DJL version 0.28.0 or patch to DJL Large Model Inference containers version 0.27.0.


CVE-2024-38459: langchain_experimental (aka LangChain Experimental) before 0.0.61 for LangChain provides Python REPL access without an o

Summary: A security vulnerability in LangChain Experimental (a Python library for building AI applications) before version 0.0.61 allows users to access a Python REPL (read-eval-print loop, an interactive environment where code can be run directly) without requiring explicit permission. This issue happened because a previous attempt to fix a related vulnerability (CVE-2024-27444) was incomplete.

Solution: Update langchain_experimental to version 0.0.61 or later. A patch is available in the commit ce0b0f22a175139df8f41cdcfb4d2af411112009 and the version comparison between 0.0.60 and 0.0.61 shows the fix.


GitHub Copilot Chat: From Prompt Injection to Data Exfiltration

Summary: GitHub Copilot Chat, a VS Code extension that lets users ask questions about their code by sending it to an AI model, was vulnerable to prompt injection (tricking an AI by hiding instructions in its input) attacks. When analyzing untrusted source code, attackers could embed malicious instructions in the code itself, which would be sent to the AI and potentially lead to data exfiltration (unauthorized copying of sensitive information).

GitHub Copilot Chat
GitHub
OpenAI
GPT-4
VS Code


CVE-2024-0103: NVIDIA Triton Inference Server for Linux contains a vulnerability where a user may cause an incorrect Initialization of

Summary: CVE-2024-0103 is a vulnerability in NVIDIA Triton Inference Server for Linux where incorrect initialization of resources caused by network issues could allow a user to disclose sensitive information. The vulnerability has a CVSS 4.0 severity rating, which measures the seriousness of security flaws on a scale of 0-10.


CVE-2024-0095: NVIDIA Triton Inference Server for Linux and Windows contains a vulnerability where a user can inject forged logs and ex

Summary: CVE-2024-0095 is a vulnerability in NVIDIA Triton Inference Server (software that runs AI models) for Linux and Windows that allows users to inject fake log entries and commands, potentially leading to code execution (running unauthorized programs), denial of service (making the system unavailable), privilege escalation (gaining higher access rights), information disclosure (exposing sensitive data), and data tampering (modifying information). The vulnerability stems from improper neutralization of log output, meaning the system doesn't properly sanitize or clean user input before adding it to logs.


CVE-2024-37014: Langflow through 0.6.19 allows remote code execution if untrusted users are able to reach the "POST /api/v1/custom_compo

Summary: Langflow versions up to 0.6.19 have a vulnerability that allows remote code execution (RCE, where attackers can run commands on a system they don't own) if untrusted users can access a specific API endpoint called POST /api/v1/custom_component and submit Python code through it. The vulnerability stems from code injection (CWE-94, where malicious code is inserted into a program), which happens because the application does not properly control how user-provided Python scripts are executed.


Why work at the EU AI Office?

Summary: This article describes the EU AI Office, a newly established regulatory organization within the European Commission tasked with enforcing the AI Act (the world's first comprehensive binding AI regulation) across the European Union. Unlike other AI safety institutes in other countries, the EU AI Office has actual enforcement powers to require AI model providers to fix problems or remove non-compliant models from the market. The office will conduct model evaluations, investigate violations, and work with international partners to shape global AI governance standards.


CVE-2024-5206: A sensitive data leakage vulnerability was identified in scikit-learn's TfidfVectorizer, specifically in versions up to

Summary: A vulnerability in scikit-learn's TfidfVectorizer (a tool that converts text into numerical data for machine learning) stored all words from training data in an attribute called `stop_words_`, instead of just the necessary ones, potentially leaking sensitive information like passwords or keys. The vulnerability affected versions up to 1.4.1.post1 but the risk depends on what type of data is being processed.

Solution: Fixed in version 1.5.0.


CVE-2024-5187: A vulnerability in the `download_model_with_test_data` function of the onnx/onnx framework, version 1.16.0, allows for a

Summary: A vulnerability in the ONNX framework (version 1.16.0) allows attackers to overwrite any file on a system by uploading a malicious tar file (a compressed archive format) with specially crafted paths. Because the vulnerable function doesn't check whether file paths are safe before extracting the tar file, attackers could potentially execute malicious code, delete important files, or compromise system security.


CVE-2024-4888: BerriAI's litellm, in its latest version, is vulnerable to arbitrary file deletion due to improper input validation on t

Summary: BerriAI's litellm has a vulnerability (CVE-2024-4888) where the `/audio/transcriptions` endpoint improperly validates user input, allowing attackers to delete arbitrary files on the server without authorization. The flaw occurs because the code uses `os.remove()` (a function that deletes files) directly on user-supplied file paths, potentially exposing sensitive files like SSH keys or databases.


CVE-2024-3234: The gaizhenbiao/chuanhuchatgpt application is vulnerable to a path traversal attack due to its use of an outdated gradio

Summary: The gaizhenbiao/chuanhuchatgpt application has a path traversal vulnerability (a flaw that lets attackers access files outside their allowed directory) because it uses an outdated version of gradio (a library for building AI interfaces). This vulnerability allows attackers to bypass security restrictions and read sensitive files like `config.json` that contain API keys (secret credentials for accessing services).

Solution: A fixed version of chuanhuchatgpt was released on 20240305 (March 5, 2024). Users should upgrade to this version or later to resolve the vulnerability.


CVE-2024-3099: A vulnerability in mlflow/mlflow version 2.11.1 allows attackers to create multiple models with the same name by exploit

Summary: MLflow version 2.11.1 has a vulnerability where attackers can create multiple models with the same name by using URL encoding (a technique that converts special characters into a format safe for web addresses). This allows attackers to cause denial of service (making a service unavailable) or data poisoning (inserting corrupted or malicious data), where an authenticated user might accidentally use a fake model instead of the real one because the system treats URL-encoded and regular names as different.


CVE-2024-3095: A Server-Side Request Forgery (SSRF) vulnerability exists in the Web Research Retriever component of langchain-ai/langch

Summary: A Server-Side Request Forgery vulnerability (SSRF, a flaw that lets attackers trick a server into making requests to unintended targets) exists in langchain version 0.1.5's Web Research Retriever component, which fails to block requests to local network addresses. This allows attackers to scan ports, access local services, read cloud metadata, and potentially execute arbitrary code (run commands on a system they don't own) by exploiting internal APIs.

langchain-ai/langchain
Web Research Retriever


CVE-2024-2928: A Local File Inclusion (LFI) vulnerability was identified in mlflow/mlflow, specifically in version 2.9.2, which was fix

Summary: A Local File Inclusion vulnerability (LFI, a flaw that lets attackers read files they shouldn't access) was found in MLflow version 2.9.2. The bug exists because the application doesn't properly check the fragment part of web addresses (the section after the '#' symbol) for directory traversal sequences like '../', which allow attackers to navigate folders and read sensitive files like system password files.

Solution: The vulnerability was fixed in version 2.11.3.


CVE-2024-0520: A vulnerability in mlflow/mlflow version 8.2.1 allows for remote code execution due to improper neutralization of specia

Summary: MLflow version 8.2.1 has a command injection vulnerability (a flaw where attackers can execute arbitrary commands by inserting malicious code into a system command) in its HTTP dataset loading function. When loading datasets, the software doesn't properly clean up filenames from URLs, allowing attackers to write files anywhere on the system and potentially run harmful commands.

Solution: The issue is fixed in version 2.9.0.


CVE-2024-5452: A remote code execution (RCE) vulnerability exists in the lightning-ai/pytorch-lightning library version 2.2.1 due to im

Summary: PyTorch Lightning version 2.2.1 has a critical vulnerability where attackers can execute arbitrary code on self-hosted applications by crafting malicious serialized data (deepdiff.Delta objects, which are used to represent changes to data). The vulnerability exists because the application doesn't properly block access to dunder attributes (special Python attributes starting with underscores), allowing attackers to bypass security restrictions and modify the application's state.


CVE-2024-4941: A local file inclusion vulnerability exists in the JSON component of gradio-app/gradio version 4.25. The vulnerability a

Summary: Gradio version 4.25 has a local file inclusion vulnerability (a security flaw where attackers can read files they shouldn't access) in its JSON component. The problem occurs because the `postprocess()` function doesn't properly validate user input before parsing it as JSON, and if the JSON contains a `path` key, the system automatically moves that file to a temporary directory where attackers can retrieve it using the `/file=..` endpoint.


CVE-2024-4325: A Server-Side Request Forgery (SSRF) vulnerability exists in the gradio-app/gradio version 4.21.0, specifically within t

Summary: A Server-Side Request Forgery vulnerability (SSRF, where a server can be tricked into making requests to unintended locations) exists in Gradio version 4.21.0 in the `/queue/join` endpoint and `save_url_to_cache` function. The vulnerability occurs because user-supplied URL input is not properly validated before being used to make HTTP requests, allowing attackers to access internal networks or sensitive cloud server information.


CVE-2024-5184: The EmailGPT service contains a prompt injection vulnerability. The service uses an API service that allows a malicious

Summary: EmailGPT has a prompt injection vulnerability (a technique where attackers hide malicious instructions in their input to trick the AI) that allows anyone with access to the service to manipulate it into leaking its internal system prompts or executing unintended commands. Attackers can exploit this by submitting specially crafted requests that trick the service into providing harmful information or performing actions it wasn't designed to do.


CVE-2024-4254: The 'deploy-website.yml' workflow in the gradio-app/gradio repository, specifically in the 'main' branch, is vulnerable

Summary: A workflow file (a set of automated tasks) in the Gradio project has a security flaw where it runs code from external copies of the repository without proper safety checks, allowing attackers to steal sensitive secrets (like API keys and authentication tokens). This happens because the workflow trusts and executes code from forks (unauthorized copies of the project) in an environment that has access to the main repository's secrets.


CVE-2024-37061: Remote Code Execution can occur in versions of the MLflow platform running version 1.11.0 or newer, enabling a malicious

Summary: CVE-2024-37061 is a remote code execution vulnerability (the ability for an attacker to run commands on someone else's system) in MLflow (a machine learning platform) version 1.11.0 and newer. An attacker can create a malicious MLproject file that executes arbitrary code when a user runs it on their computer.


CVE-2024-37060: Deserialization of untrusted data can occur in versions of the MLflow platform running version 1.27.0 or newer, enabling

Summary: CVE-2024-37060 is a vulnerability in MLflow (a machine learning platform) version 1.27.0 and newer where deserialization of untrusted data (the process of converting received data back into usable objects without checking if it's safe) can occur. A malicious Recipe (a workflow template in MLflow) could exploit this to execute arbitrary code (run any commands) on a user's computer when the Recipe is run.


CVE-2024-37059: Deserialization of untrusted data can occur in versions of the MLflow platform running version 0.5.0 or newer, enabling

Summary: CVE-2024-37059 is a vulnerability in MLflow (a platform for managing machine learning workflows) version 0.5.0 and newer where deserialization of untrusted data (converting data from an external format into usable code without verifying it's safe) can occur. An attacker can upload a malicious PyTorch model (a type of machine learning model file) that executes arbitrary code (runs any commands they choose) on a user's computer when the model is opened or used.


CVE-2024-37058: Deserialization of untrusted data can occur in versions of the MLflow platform running version 2.5.0 or newer, enabling

Summary: CVE-2024-37058 is a vulnerability in MLflow (a platform for managing machine learning workflows) version 2.5.0 and newer that allows deserialization of untrusted data (the process of converting data from storage into usable objects without checking if it's safe). An attacker can upload a malicious Langchain AgentExecutor model (a type of AI component) that runs arbitrary code on a user's system when that user interacts with it.


CVE-2024-37057: Deserialization of untrusted data can occur in versions of the MLflow platform running version 2.0.0rc0 or newer, enabli

Summary: CVE-2024-37057 is a vulnerability in MLflow (an open-source machine learning platform) versions 2.0.0rc0 and newer that allows deserialization of untrusted data (converting data from an untrusted source back into executable code). An attacker could upload a malicious TensorFlow model (a type of machine learning model) that runs arbitrary code (any commands an attacker chooses) on a user's computer when the model is loaded or used.


CVE-2024-37056: Deserialization of untrusted data can occur in versions of the MLflow platform running version 1.23.0 or newer, enabling

Summary: CVE-2024-37056 is a vulnerability in MLflow (a machine learning platform) version 1.23.0 and newer that allows deserialization of untrusted data (loading and executing code from data that hasn't been verified as safe). An attacker can upload a malicious LightGBM or scikit-learn model (machine learning libraries) that runs arbitrary code (any commands the attacker chooses) on a user's computer when the model is opened.


CVE-2024-37055: Deserialization of untrusted data can occur in versions of the MLflow platform running version 1.24.0 or newer, enabling

Summary: CVE-2024-37055 is a vulnerability in MLflow (a machine learning platform) versions 1.24.0 and newer where deserialization of untrusted data (the process of converting saved data back into usable objects without checking if it's safe) can occur. This allows an attacker to upload a malicious pmdarima model (a machine learning model for time-series forecasting) that runs arbitrary code (any commands the attacker chooses) on a user's computer when the model is loaded and used.


CVE-2024-37054: Deserialization of untrusted data can occur in versions of the MLflow platform running version 0.9.0 or newer, enabling

Summary: CVE-2024-37054 is a vulnerability in MLflow (a machine learning platform) version 0.9.0 and newer that allows deserialization of untrusted data (unsafe processing of data from untrusted sources). An attacker can upload a malicious PyFunc model (a machine learning model format) that runs arbitrary code (any commands an attacker wants) on a user's computer when the model is used.


CVE-2024-37053: Deserialization of untrusted data can occur in versions of the MLflow platform running version 1.1.0 or newer, enabling

Summary: CVE-2024-37053 is a vulnerability in MLflow (a machine learning platform) version 1.1.0 and newer where deserialization of untrusted data (the process of converting saved data back into usable code without checking if it's safe) can occur. An attacker can upload a malicious scikit-learn model (a machine learning library) that runs arbitrary code (any commands the attacker chooses) on a user's computer when the model is loaded and used.


CVE-2024-37052: Deserialization of untrusted data can occur in versions of the MLflow platform running version 1.1.0 or newer, enabling

Summary: CVE-2024-37052 is a vulnerability in MLflow (a machine learning platform) version 1.1.0 and newer where deserialization of untrusted data (converting data from an external format back into code without checking if it's safe) allows a malicious scikit-learn model (a machine learning library) to execute arbitrary code on a user's system when the model is loaded and used. This means an attacker could upload a harmful model that runs malicious commands when someone interacts with it.


CVE-2024-37065: Deserialization of untrusted data can occur in versions 0.6 or newer of the skops python library, enabling a maliciously

Summary: CVE-2024-37065 is a vulnerability in skops (a Python library) version 0.6 and newer where deserialization (the process of converting saved data back into usable code) of untrusted data can occur, allowing a maliciously crafted model file to run arbitrary code on a user's computer when loaded.


CVE-2024-4253: A command injection vulnerability exists in the gradio-app/gradio repository, specifically within the 'test-functional.y

Summary: A command injection vulnerability (a type of attack where specially crafted input tricks a system into running unintended commands) exists in the Gradio project's automated workflow file, where unsanitized (unfiltered) repository and branch names could be exploited to steal sensitive credentials like authentication tokens. The vulnerability affects Gradio versions up to @gradio/video@0.6.12.


CVE-2024-3829: qdrant/qdrant version 1.9.0-dev is vulnerable to arbitrary file read and write during the snapshot recovery process. Att

Summary: Qdrant version 1.9.0-dev has a vulnerability in its snapshot recovery process (a feature that restores a database from a backup) that allows attackers to read and write arbitrary files on the server by inserting symlinks (shortcuts to other files) into snapshot files. This could potentially give attackers complete control over the system.

Solution: Update to version v1.9.0, where the issue is fixed.


CVE-2024-5565: The Vanna library uses a prompt function to present the user with visualized results, it is possible to alter the prompt

Summary: The Vanna library (a tool for generating data visualizations) has a vulnerability where attackers can use prompt injection (tricking an AI by hiding instructions in its input) to alter how the library processes user requests and run arbitrary Python code instead of creating the intended visualization. This happens when external input is sent to the library's 'ask' method with visualization enabled, which is the default setting, leading to remote code execution (attackers being able to run commands on a system they don't own).


CVE-2024-37032: Ollama before 0.1.34 does not validate the format of the digest (sha256 with 64 hex digits) when getting the model path,

Summary: Ollama versions before 0.1.34 have a security flaw where they don't properly check the format of digests (sha256 hashes that should be exactly 64 hexadecimal digits) when looking up model file paths. This allows attackers to bypass security checks by using invalid digest formats, such as ones with too few digits, too many digits, or paths starting with '../' (a path traversal technique that accesses files outside the intended directory).

Solution: Update Ollama to version 0.1.34 or later. The fix is available in the release notes at https://github.com/ollama/ollama/compare/v0.1.33...v0.1.34 and was implemented in pull request #4175.


CVE-2024-3924: A code injection vulnerability exists in the huggingface/text-generation-inference repository, specifically within the `

Summary: A code injection vulnerability (injecting malicious code into a system) exists in the huggingface/text-generation-inference repository's workflow file, where user input from GitHub branch names is unsafely used to build commands. An attacker can exploit this by creating a malicious branch name and submitting a pull request, potentially executing arbitrary code on the GitHub Actions runner (the automated system that runs tests and builds for the project).

Solution: This issue was fixed in version 2.0.0. Users should update to version 2.0.0 or later.


CVE-2024-3584: qdrant/qdrant version 1.9.0-dev is vulnerable to path traversal due to improper input validation in the `/collections/{n

Summary: Qdrant version 1.9.0-dev has a path traversal vulnerability (a security flaw where an attacker manipulates file paths to access unintended locations) in its snapshot upload endpoint that allows attackers to write files anywhere on the server by encoding special characters in the request. This could lead to complete system compromise through arbitrary file upload and overwriting.

Solution: The issue is fixed in version 1.9.0. Users should upgrade to this version or later.


CVE-2024-5185: The EmbedAI application is susceptible to security issues that enable Data Poisoning attacks. This weakness could result

Summary: EmbedAI has a security flaw that allows data poisoning attacks (injecting false or harmful information into an AI system) through a CSRF vulnerability (cross-site request forgery, where an attacker tricks a user into performing unwanted actions on a website they're logged into). An attacker can direct users to a malicious webpage that exploits weak session management and CORS policies (which control what external websites can access the application), tricking them into uploading bad data that corrupts the application's language model.


Automatic Tool Invocation when Browsing with ChatGPT - Threats and Mitigations

Summary: ChatGPT's browsing tool can be tricked into automatically invoking other tools (like image creation or memory management) when users visit websites containing hidden instructions, a vulnerability known as prompt injection (tricking an AI by hiding instructions in its input). While OpenAI added some protections, minor prompting tricks can bypass them, and this issue affects other AI applications as well.

Solution: For custom GPTs with AI Actions, creators can use the x-openai-isConsequential flag as a mitigation to put users in control, though the source notes this approach 'still lacks a great user experience, like better visualization to understand what the action is about to do.'


CVE-2024-4858: The Testimonial Carousel For Elementor plugin for WordPress is vulnerable to unauthorized modification of data due to a

Summary: The Testimonial Carousel For Elementor WordPress plugin (versions up to 10.2.0) has a missing authorization check in the 'save_testimonials_option_callback' function, allowing unauthenticated attackers to modify data like OpenAI API keys without permission. This vulnerability is classified as CWE-862 (missing authorization, where a system doesn't verify that a user has permission to perform an action).


Robust governance for the AI Act: Insights and highlights from Novelli et al. (2024)

Summary: This overview discusses the European AI Act and the governance framework needed to implement it, focusing on the European Commission's responsibilities and the AI Office. Key tasks include establishing guidelines for classifying high-risk AI systems, defining what counts as significant modifications (changes that alter a system's risk level), and setting standards for transparency and enforcement across EU member states.

Solution: The source suggests that the Commission should adopt 'predetermined change management plans akin to those in medicine' to assess modifications to AI systems. These plans would be documents outlining anticipated changes (such as performance adjustments or shifts in intended use) and the methods for evaluating whether those changes substantially alter the system's risk level. The source also recommends that standard fine-tuning of foundation models (training adjustments to pre-existing AI models) should not be considered a significant modification unless safety layers are removed or other actions clearly increase risk.


ChatGPT: Hacking Memories with Prompt Injection

Summary: ChatGPT's new memory feature, which lets the AI remember information across different chat sessions for a more personalized experience, can be exploited through indirect prompt injection (tricking an AI by hiding malicious instructions in its input). Attackers could manipulate ChatGPT into storing false information, biases, or unwanted instructions by injecting commands through connected apps like Google Drive, uploaded documents, or web browsing features.


CVE-2024-0453: The AI ChatBot plugin for WordPress is vulnerable to unauthorized modification of data due to a missing capability check

Summary: The AI ChatBot plugin for WordPress (up to version 5.3.4) has a security flaw where a function called openai_file_delete_callback lacks a capability check (verification that a user has permission to perform an action). This allows any authenticated user with subscriber-level access or higher to delete files from a connected OpenAI account without proper authorization.


CVE-2024-0452: The AI ChatBot plugin for WordPress is vulnerable to unauthorized modification of data due to a missing capability check

Summary: The AI ChatBot plugin for WordPress (up to version 5.3.4) has a missing capability check (a missing authorization check that verifies user permissions) in its file upload function, allowing authenticated users with basic subscriber access to upload files to a connected OpenAI account without proper permission verification. This vulnerability affects all versions through 5.3.4 and could let low-privilege attackers modify data on the linked OpenAI account.


CVE-2024-0451: The AI ChatBot plugin for WordPress is vulnerable to unauthorized access of data due to a missing capability check on th

Summary: The AI ChatBot plugin for WordPress has a security flaw in versions up to 5.3.4 where a function lacks a capability check (a security control that verifies a user has permission to perform an action). This allows authenticated users with subscriber-level access or higher to view files stored in a connected OpenAI account without authorization.

Solution: A patch is available at https://plugins.trac.wordpress.org/changeset/3089461/chatbot/trunk/includes/openai/qcld-bot-openai.php. Users should update their AI ChatBot plugin to a version after 5.3.4.


Machine Learning Attack Series: Backdooring Keras Models and How to Detect It

Summary: This post examines how attackers can insert hidden malicious code into machine learning models (a technique called backdooring) through supply chain attacks, specifically targeting Keras models (a popular framework for building AI systems). The authors demonstrate this attack and then explore tools that can detect when a model has been compromised in this way.


CVE-2024-4263: A broken access control vulnerability exists in mlflow/mlflow versions before 2.10.1, where low privilege users with onl

Summary: MLflow (a tool for managing machine learning experiments) versions before 2.10.1 have a broken access control vulnerability where users with only EDIT permissions can delete artifacts (saved files or data from experiments) they shouldn't be able to delete. The bug happens because the system doesn't properly check permissions when users request to delete artifacts, even though the documentation says EDIT users should only be able to read and update, not delete.

Solution: Update mlflow to version 2.10.1 or later.


CVE-2024-3848: A path traversal vulnerability exists in mlflow/mlflow version 2.11.0, identified as a bypass for the previously address

Summary: MLflow version 2.11.0 has a path traversal vulnerability (a security flaw where an attacker can access files outside intended directories) that bypasses a previous fix. An attacker can use a '#' character in artifact URLs to skip validation and read sensitive files like SSH keys and cloud credentials from the server's filesystem. The vulnerability exists because the application doesn't properly validate the fragment portion (the part after '#') of URLs before converting them to filesystem paths.


CVE-2024-4181: A command injection vulnerability exists in the RunGptLLM class of the llama_index library, version 0.9.47, used by the

Summary: A command injection vulnerability (a flaw that lets attackers run unauthorized commands) exists in the RunGptLLM class of the llama_index library version 0.9.47, which connects applications to language models. The vulnerability uses the eval function (a tool that executes text as code) unsafely, potentially allowing a malicious LLM provider to run arbitrary commands and take control of a user's machine.

Solution: This issue was fixed in version 0.10.13 of the llama_index library. Users should upgrade to version 0.10.13 or later.


CVE-2024-34440: Unrestricted Upload of File with Dangerous Type vulnerability in Jordy Meow AI Engine: ChatGPT Chatbot.This issue affect

Summary: CVE-2024-34440 is an unrestricted file upload vulnerability (a security flaw that lets users upload files without proper checks on file type) in the Jordy Meow AI Engine: ChatGPT Chatbot plugin affecting versions through 2.2.63. This vulnerability could potentially allow attackers to upload dangerous files to a system, but no severity score has been assigned yet.

Jordy Meow AI Engine
ChatGPT Chatbot WordPress Plugin


CVE-2024-0100: NVIDIA Triton Inference Server for Linux contains a vulnerability in the tracing API, where a user can corrupt system fi

Summary: CVE-2024-0100 is a vulnerability in NVIDIA Triton Inference Server for Linux that allows a user to corrupt system files through the tracing API (a feature that tracks how the server runs). Successfully exploiting this vulnerability could cause denial of service (making the system unavailable) and data tampering (unauthorized changes to data).


CVE-2024-0088: NVIDIA Triton Inference Server for Linux contains a vulnerability in shared memory APIs, where a user can cause an impro

Summary: CVE-2024-0088 is a vulnerability in NVIDIA Triton Inference Server for Linux where a network user can trigger improper memory access through shared memory APIs, potentially causing denial of service (making a service unavailable) or data tampering. The vulnerability stems from out-of-bounds write errors, meaning the software tries to write data to memory locations it shouldn't access.


CVE-2024-0087: NVIDIA Triton Inference Server for Linux contains a vulnerability where a user can set the logging location to an arbitr

Summary: CVE-2024-0087 is a vulnerability in NVIDIA Triton Inference Server for Linux that allows a user to set the logging location to any file they choose, and if that file already exists, logs get added to it. This could allow an attacker to execute code, crash the system, gain elevated permissions, steal information, or modify data.

confidentiality integrity availability safety


CVE-2024-34359: llama-cpp-python is the Python bindings for llama.cpp. `llama-cpp-python` depends on class `Llama` in `llama.py` to load

Summary: llama-cpp-python (Python bindings for llama.cpp, a tool for running AI models locally) has a vulnerability where it loads chat templates from model files without proper security checks. When these templates are processed using Jinja2 (a templating engine), an attacker can inject malicious code through a specially crafted model file, leading to remote code execution (the ability to run arbitrary commands on the victim's computer).


CVE-2024-34527: spaces_plugin/app.py in SolidUI 0.4.0 has an unnecessary print statement for an OpenAI key. The printed string might be

Summary: SolidUI version 0.4.0 contains a bug where the file spaces_plugin/app.py has an unnecessary print statement that outputs an OpenAI key (a secret credential used to authenticate with OpenAI's services). This printed key could be captured in log files (records of system activity), potentially exposing the credential to unauthorized users.


CVE-2024-34510: Gradio before 4.20 allows credential leakage on Windows.

Summary: Gradio (a framework for building web interfaces for machine learning models) before version 4.20 has a vulnerability on Windows where credentials can be unintentionally revealed. The issue stems from improper encoding or escaping of output (meaning the software doesn't properly clean or protect sensitive information before displaying it).

Solution: Update Gradio to version 4.20 or later.


CVE-2024-34073: sagemaker-python-sdk is a library for training and deploying machine learning models on Amazon SageMaker. In affected ve

Summary: A vulnerability in sagemaker-python-sdk (a library for machine learning on Amazon SageMaker) allows OS command injection (running unauthorized system commands) if unsafe input is passed to the capture_dependencies function's requirements_path parameter, potentially letting attackers execute code remotely or disrupt service. The vulnerability affects versions before 2.214.3.

Solution: Upgrade to version 2.214.3 or later. Alternatively, users unable to upgrade should not override the "requirements_path" parameter of the capture_dependencies function and instead use the default value.


CVE-2024-34072: sagemaker-python-sdk is a library for training and deploying machine learning models on Amazon SageMaker. The sagemaker.

Summary: A vulnerability in the sagemaker-python-sdk library (used for machine learning on Amazon SageMaker) allows unsafe deserialization, where the NumpyDeserializer module can execute malicious code if it processes untrusted pickled data (serialized Python objects stored in a binary format). An attacker could exploit this to run arbitrary commands on a system or crash it.

Solution: Upgrade to sagemaker-python-sdk version 2.218.0 or later. If unable to upgrade, do not process pickled numpy object arrays from untrusted sources or data that could have been modified by others. Only use pickled numpy object arrays from sources you trust.


CVE-2023-5675: A flaw was found in Quarkus. When a Quarkus RestEasy Classic or Reactive JAX-RS endpoint has its methods declared in the

Summary: CVE-2023-5675 is a security flaw in Quarkus (a Java framework for building applications) where authorization checks are bypassed for REST API endpoints whose methods are defined in abstract classes or modified by extensions using annotation processors, if certain security settings are enabled. This means unauthorized users could potentially access protected API endpoints that should require authentication or specific permissions.

Solution: N/A -- no mitigation discussed in source.


CVE-2024-31584: Pytorch before v2.2.0 has an Out-of-bounds Read vulnerability via the component torch/csrc/jit/mobile/flatbuffer_loader.

Summary: PyTorch versions before 2.2.0 contain an out-of-bounds read vulnerability (a bug where code tries to read data from memory outside its allowed range) in the flatbuffer_loader component, which is used for loading machine learning models on mobile devices. This vulnerability could potentially allow attackers to read sensitive information from memory or cause the program to crash.

Solution: Upgrade to PyTorch version 2.2.0 or later. A patch is available at https://github.com/pytorch/pytorch/commit/7c35874ad664e74c8e4252d67521f3986eadb0e6.


CVE-2024-31583: Pytorch before version v2.2.0 was discovered to contain a use-after-free vulnerability in torch/csrc/jit/mobile/interpre

Summary: PyTorch versions before v2.2.0 contain a use-after-free vulnerability (a memory bug where code tries to access data that has already been freed) in the mobile interpreter component. This vulnerability was identified in the torch/csrc/jit/mobile/interpreter.cpp file.

Solution: Update PyTorch to version v2.2.0 or later. A patch is available at https://github.com/pytorch/pytorch/commit/9c7071b0e324f9fb68ab881283d6b8d388a4bcd2.


CVE-2024-31580: PyTorch before v2.2.0 was discovered to contain a heap buffer overflow vulnerability in the component /runtime/vararg_fu

Summary: PyTorch versions before v2.2.0 contain a heap buffer overflow vulnerability (a type of memory safety bug where a program writes data beyond allocated memory limits) in its runtime component that allows attackers to crash the software through specially crafted input. This is a Denial of Service attack, meaning the goal is to make the software unusable rather than steal data.

Solution: Upgrade to PyTorch v2.2.0 or later. A patch is available at https://github.com/pytorch/pytorch/commit/b5c3a17c2c207ebefcb85043f0cf94be9b2fef81.


CVE-2024-3660: A arbitrary code injection vulnerability in TensorFlow's Keras framework (<2.13) allows attackers to execute arbitrary c

Summary: CVE-2024-3660 is a code injection vulnerability (a flaw that lets attackers insert and run harmful code) in TensorFlow's Keras framework (a machine learning library) affecting versions before 2.13. Attackers can exploit this to execute arbitrary code (run commands they choose) with the same permissions as the application using a vulnerable model.


CVE-2024-3573: mlflow/mlflow is vulnerable to Local File Inclusion (LFI) due to improper parsing of URIs, allowing attackers to bypass

Summary: MLflow (a machine learning platform) has a vulnerability where its URI parsing function incorrectly classifies certain file paths as non-local, allowing attackers to read sensitive files they shouldn't access. By crafting malicious model versions with specially crafted parameters, attackers can bypass security checks and read arbitrary files from the system.


CVE-2024-3571: langchain-ai/langchain is vulnerable to path traversal due to improper limitation of a pathname to a restricted director

Summary: LangChain's LocalFileStore feature has a path traversal vulnerability (a security flaw where attackers can access files outside the intended directory by using special path sequences like '../'). An attacker can exploit this to read or write any files on the system, potentially stealing data or executing malicious code. The problem stems from the mset and mget methods not properly filtering user input before handling file paths.


CVE-2024-2912: An insecure deserialization vulnerability exists in the BentoML framework, allowing remote code execution (RCE) by sendi

Summary: BentoML (a framework for building AI applications) contains an insecure deserialization vulnerability that lets attackers run arbitrary commands on servers by sending specially crafted requests. When the framework deserializes (converts stored data back into usable objects) a malicious object, it automatically executes hidden OS commands, giving attackers control of the server.


CVE-2024-1594: A path traversal vulnerability exists in the mlflow/mlflow repository, specifically within the handling of the `artifact

Summary: CVE-2024-1594 is a path traversal vulnerability (a flaw that lets attackers access files outside their permitted directory) in MLflow's experiment creation feature. Attackers can exploit this by inserting a fragment component (#) into the artifact_location parameter to read arbitrary files on the server.


CVE-2024-1593: A path traversal vulnerability exists in the mlflow/mlflow repository due to improper handling of URL parameters. By smu

Summary: MLflow, a machine learning platform, has a path traversal vulnerability (a security flaw where attackers can access files outside intended directories) caused by improper handling of URL parameters. Attackers can use the semicolon (;) character to hide malicious path sequences in URLs, potentially gaining unauthorized access to sensitive files or compromising the server.


CVE-2024-1561: An issue was discovered in gradio-app/gradio, where the `/component_server` endpoint improperly allows the invocation of

Summary: Gradio, a popular Python library for building AI interfaces, has a vulnerability in its `/component_server` endpoint that lets attackers call any method on a Component class with their own arguments. By exploiting a specific method called `move_resource_to_block_cache()`, attackers can copy files from the server's filesystem to a temporary folder and download them, potentially exposing sensitive data like API keys, especially when apps are shared online or hosted on platforms like Hugging Face.


CVE-2024-1560: A path traversal vulnerability exists in the mlflow/mlflow repository, specifically within the artifact deletion functio

Summary: A path traversal vulnerability (a security flaw where attackers use special characters like ../ to access files outside their intended directory) exists in MLflow's artifact deletion feature. Attackers can delete arbitrary files on a server by exploiting an extra decoding step that fails to properly validate user input, and this vulnerability affects versions up to 2.9.2.


CVE-2024-1558: A path traversal vulnerability exists in the `_create_model_version()` function within `server/handlers.py` of the mlflo

Summary: CVE-2024-1558 is a path traversal vulnerability (a security flaw where an attacker uses special characters like "../" to access files outside their intended directory) in MLflow's model version creation function. An attacker can craft a malicious `source` parameter that bypasses the validation check, allowing them to read any file on the server when fetching model artifacts.


CVE-2024-1483: A path traversal vulnerability exists in mlflow/mlflow version 2.9.2, allowing attackers to access arbitrary files on th

Summary: CVE-2024-1483 is a path traversal vulnerability (a weakness that lets attackers access files outside intended directories) in MLflow version 2.9.2 that allows attackers to read arbitrary files on a server. The vulnerability occurs because the server doesn't properly validate user input in the 'artifact_location' and 'source' parameters, and attackers can exploit this by sending specially crafted HTTP POST requests that use '#' instead of '?' in local URIs to navigate the server's directory structure.


CVE-2024-1183: An SSRF (Server-Side Request Forgery) vulnerability exists in the gradio-app/gradio repository, allowing attackers to sc

Summary: CVE-2024-1183 is an SSRF vulnerability (a flaw where an attacker tricks a server into making requests to internal networks) in the Gradio application that lets attackers scan and identify open ports on internal networks by manipulating the 'file' parameter in requests and reading responses for specific headers or error messages.

Solution: A patch is available at https://github.com/gradio-app/gradio/commit/2ad3d9e7ec6c8eeea59774265b44f11df7394bb4


Bobby Tables but with LLM Apps - Google NotebookLM Data Exfiltration

Summary: Google's NotebookLM is a tool that lets users upload files for an AI to analyze, but it's vulnerable to prompt injection (tricking the AI by hiding instructions in uploaded files) that can manipulate the AI's responses and expose what users see. The tool also has a data exfiltration vulnerability (attackers stealing information) when processing untrusted files, and there is currently no known way to prevent these attacks, meaning users cannot fully trust the AI's responses when working with files from unknown sources.


CVE-2024-31462: stable-diffusion-webui is a web interface for Stable Diffusion, implemented using Gradio library. Stable-diffusion-webui

Summary: Stable-diffusion-webui version 1.7.0 has a vulnerability where user input from the Backup/Restore tab is not properly validated before being used to create file paths, allowing attackers to write JSON files to arbitrary locations on Windows systems where the web server has access. This is a limited file write vulnerability (a security flaw that lets attackers create or modify files in unintended locations) that could let an attacker place malicious files on the server.

Stability AI
stable-diffusion-webui
Gradio


CVE-2023-51409: Unrestricted Upload of File with Dangerous Type vulnerability in Jordy Meow AI Engine: ChatGPT Chatbot.This issue affect

Summary: CVE-2023-51409 is a vulnerability in the Jordy Meow AI Engine: ChatGPT Chatbot plugin (versions up to 1.9.98) that allows unrestricted upload of dangerous file types, meaning attackers can upload files that shouldn't be allowed without proper validation. This vulnerability could potentially lead to remote code execution (running malicious commands on the affected system).

Jordy Meow AI Engine
ChatGPT Chatbot plugin


CVE-2024-3568: The huggingface/transformers library is vulnerable to arbitrary code execution through deserialization of untrusted data

Summary: The huggingface/transformers library has a vulnerability where attackers can run arbitrary code on a victim's machine by tricking them into loading a malicious checkpoint file. The problem occurs in the `load_repo_checkpoint()` function, which uses `pickle.load()` (a Python function that reconstructs objects from serialized data) on data that might come from untrusted sources, allowing attackers to execute remote code execution (RCE, where an attacker runs commands on a system they don't own).


CVE-2024-2221: qdrant/qdrant is vulnerable to a path traversal and arbitrary file upload vulnerability via the `/collections/{COLLECTIO

Summary: Qdrant (a vector database software) has a vulnerability in its snapshot upload endpoint that allows attackers to upload files to any location on the server's filesystem through path traversal (using special file path sequences to access directories they shouldn't). This could let attackers execute arbitrary code on the server and damage the system's integrity and availability.

Solution: A patch is available at https://github.com/qdrant/qdrant/commit/e6411907f0ecf3c2f8ba44ab704b9e4597d9705d


CVE-2024-1728: gradio-app/gradio is vulnerable to a local file inclusion vulnerability due to improper validation of user-supplied inpu

Summary: Gradio (a framework for building AI interfaces) has a vulnerability in its UploadButton component where it doesn't properly validate (check) user input, allowing attackers to read any file on the server by manipulating file paths sent to the `/queue/join` endpoint. This could let attackers steal sensitive files like SSH keys (credentials used for secure server access) and potentially execute arbitrary code on the system.


CVE-2024-3098: A vulnerability was identified in the `exec_utils` class of the `llama_index` package, specifically within the `safe_eva

Summary: A vulnerability was found in the `safe_eval` function of the `llama_index` package that allows prompt injection (tricking an AI by hiding instructions in its input) to execute arbitrary code (running code an attacker chooses). The flaw exists because the input validation is insufficient, meaning the package doesn't properly check what data is being passed in, allowing attackers to bypass safety restrictions that were meant to prevent this type of attack.


CVE-2024-28224: Ollama before 0.1.29 has a DNS rebinding vulnerability that can inadvertently allow remote access to the full API, there

Summary: Ollama before version 0.1.29 has a DNS rebinding vulnerability (a technique where an attacker tricks a system into connecting to a malicious server by manipulating how domain names are translated into addresses), which allows unauthorized remote access to its full API. This vulnerability could let an attacker interact with the language model, remove models, or cause a denial of service (making a system unavailable by overloading it with requests).

Solution: Update Ollama to version 0.1.29 or later.


CVE-2024-31224: GPT Academic provides interactive interfaces for large language models. A vulnerability was found in gpt_academic versio

Summary: GPT Academic is a tool that provides interactive interfaces for large language models. Versions 3.64 through 3.73 have a vulnerability where the server deserializes untrusted data (processes data from users without verifying it's safe), which could allow attackers to execute code remotely on any exposed server. Any device running these vulnerable versions and accessible over the internet is at risk.

Solution: Upgrade to version 3.74, which contains a patch for the issue. The source states: 'There are no known workarounds aside from upgrading to a patched version.'


Google AI Studio Data Exfiltration via Prompt Injection - Possible Regression and Fix

Summary: Google AI Studio had a vulnerability that allowed attackers to steal data through prompt injection (tricking an AI by hiding malicious instructions in its input), where a malicious file could trick the AI into exfiltrating other uploaded files to an attacker's server via image tags. The vulnerability appeared in a recent update but was fixed within 12 days of being reported to Google on February 17, 2024.

Solution: The issue was fixed by Google and did not reproduce after the company heard back about the report 12 days later (by approximately February 29, 2024). The ticket was closed as 'Duplicate' on March 3, 2024, suggesting the vulnerability may have also been caught through internal testing.


The dangers of AI agents unfurling hyperlinks and what to do about it

Summary: Unfurling is when an application automatically expands hyperlinks to show previews, which can be exploited in AI chatbots to leak data. When an attacker uses prompt injection (tricking an AI by hiding instructions in its input) to make the chatbot generate a link containing sensitive information from earlier conversations, the unfurling feature automatically sends that data to a third-party server, potentially exposing private information.

Solution: To disable unfurling in Slack Apps, modify the message creation function to include unfurl settings in the JSON object: set "unfurl_links": False and "unfurl_media": False when creating the message, as shown in the example code: def create_message(text): message = { "text": text, "unfurl_links": False, "unfurl_media": False } return json.dumps(message)


CVE-2024-3078: A vulnerability was found in Qdrant up to 1.6.1/1.7.4/1.8.2 and classified as critical. This issue affects some unknown

Summary: A critical vulnerability was discovered in Qdrant (a vector database system) versions up to 1.6.1, 1.7.4, and 1.8.2 that allows path traversal (a technique where attackers access files outside intended directories) through the Full Snapshot REST API (a web interface for creating system backups). This flaw could let attackers manipulate file paths to access unauthorized files on the system.

Solution: Upgrade to Qdrant version 1.8.3 or later. The specific patch is identified as 3ab5172e9c8f14fa1f7b24e7147eac74e2412b62.


CVE-2024-1729: A timing attack vulnerability exists in the gradio-app/gradio repository, specifically within the login function in rout

Summary: CVE-2024-1729 is a timing attack vulnerability (where an attacker guesses a password by measuring how long the system takes to reject it) in the Gradio application's login function. The vulnerability exists because the code directly compares the entered password with the stored password using a simple equality check, which can leak information through response time differences, potentially allowing attackers to bypass authentication and gain unauthorized access.

Solution: A patch is available at https://github.com/gradio-app/gradio/commit/e329f1fd38935213fe0e73962e8cbd5d3af6e87b. Additionally, a bounty reference with more details is provided at https://huntr.com/bounties/f6a10a8d-f538-4cb7-9bb2-85d9f5708124.


CVE-2024-29100: Unrestricted Upload of File with Dangerous Type vulnerability in Jordy Meow AI Engine: ChatGPT Chatbot.This issue affect

Summary: CVE-2024-29100 is an unrestricted file upload vulnerability (a security flaw that allows attackers to upload harmful files without proper checks) in the Jordy Meow AI Engine: ChatGPT Chatbot plugin for WordPress, affecting versions up to 2.1.4. This vulnerability could potentially allow attackers to upload dangerous files to a website using this plugin.


CVE-2024-29090: Server-Side Request Forgery (SSRF) vulnerability in Jordy Meow AI Engine: ChatGPT Chatbot.This issue affects AI Engine:

Summary: A server-side request forgery (SSRF, a vulnerability where an attacker tricks a server into making unintended requests to other systems) vulnerability was found in the AI Engine: ChatGPT Chatbot plugin by Jordy Meow, affecting versions up to 2.1.4. The vulnerability allows authenticated attackers to exploit the plugin to perform unauthorized requests.


CVE-2024-1540: A command injection vulnerability exists in the deploy+test-visual.yml workflow of the gradio-app/gradio repository, due

Summary: CVE-2024-1540 is a command injection vulnerability (a weakness where an attacker can insert malicious commands into code that gets executed) in the gradio-app/gradio repository's workflow file. Attackers could exploit this by manipulating GitHub context information within expressions to run unauthorized commands, potentially stealing secrets or modifying the repository. The vulnerability stems from unsafe handling of variables that are directly substituted into scripts before execution.

Solution: Remediation involves setting untrusted input values to intermediate environment variables to prevent direct influence on script generation.


CVE-2024-2206: An SSRF vulnerability exists in the gradio-app/gradio due to insufficient validation of user-supplied URLs in the `/prox

Summary: CVE-2024-2206 is an SSRF vulnerability (server-side request forgery, where an attacker tricks a server into making requests to unintended targets) in Gradio, an AI framework. Attackers can exploit this by sending specially crafted requests with an `X-Direct-Url` header to add arbitrary URLs to a list that the application uses for proxying (forwarding) requests, potentially allowing unauthorized access to internal systems. The vulnerability exists because the application does not properly validate URLs in its `build_proxy_request` function.


CVE-2024-1455: A vulnerability in the langchain-ai/langchain repository allows for a Billion Laughs Attack, a type of XML External Enti

Summary: CVE-2024-1455 is a vulnerability in the langchain-ai/langchain repository that allows a Billion Laughs Attack, a type of XML External Entity (XXE) exploitation where an attacker nests multiple layers of entities within an XML document to make the parser consume excessive CPU and memory resources, causing a denial of service (DoS, where a system becomes unavailable to legitimate users).

Solution: A patch is available at https://github.com/langchain-ai/langchain/commit/727d5023ce88e18e3074ef620a98137d26ff92a3


The AI Office is hiring

Summary: The European Commission is hiring AI specialists to work in the AI Office, which will enforce the EU's AI Act by overseeing compliance of general-purpose AI models (large AI systems available to the public). The office will have real regulatory powers to require companies to implement safety measures, restrict models, or remove them from the market, and will develop evaluation tools and benchmarks to identify dangerous AI behaviors.


CVE-2024-1727: A Cross-Site Request Forgery (CSRF) vulnerability in gradio-app/gradio allows attackers to upload multiple large files t

Summary: CVE-2024-1727 is a CSRF vulnerability (cross-site request forgery, where an attacker tricks a victim into making unintended requests) in Gradio that lets attackers upload large files to a victim's computer without permission. An attacker can create a malicious webpage that, when visited, automatically uploads files to the victim's system, potentially filling up their disk space and causing a denial of service (making the system unusable).

Solution: A patch is available at https://github.com/gradio-app/gradio/commit/84802ee6a4806c25287344dce581f9548a99834a


The AI Office: What is it, and how does it work?

Summary: The European AI Office is a new EU regulator created to oversee general purpose AI (GPAI) models and systems, which are AI systems designed to perform a wide range of tasks, across all 27 EU Member States under the AI Act. It monitors compliance, analyzes emerging risks, develops evaluation capabilities, produces voluntary codes of practice for companies to follow, and coordinates enforcement between national regulators and international partners. The Office also supports small and medium businesses with compliance resources and oversees regulatory sandboxes, which are controlled environments where companies can test AI systems before full deployment.


CVE-2024-29037: datahub-helm provides the Kubernetes Helm charts for deploying Datahub and its dependencies on a Kubernetes cluster. Sta

Summary: A vulnerability in datahub-helm (Helm charts, which are templates for deploying applications on Kubernetes clusters) versions 0.1.143 through 0.2.181 allowed personal access tokens (credentials that grant access to the system) to be created using a publicly known default secret key instead of a random one. This meant attackers could potentially generate their own valid tokens to access DataHub instances if Metadata Service Authentication (a security feature) was enabled during a specific vulnerable time period.

Solution: Update to version 0.2.182, which contains a patch for this issue. As a workaround, reset the token signing key to be a random value, which will invalidate active personal access tokens.


CVE-2024-29018: Moby is an open source container framework that is a key component of Docker Engine, Docker Desktop, and other distribut

Summary: Moby (the container framework underlying Docker) has a bug in how it handles DNS requests from internal networks (networks isolated from external communication). When a container on an internal network needs to resolve a domain name, Moby forwards the request through the host's network namespace instead of the container's own network, which can leak data to external servers that an attacker controls. Docker Desktop is not affected by this issue.

Solution: Moby releases 26.0.0, 25.0.4, and 23.0.11 are patched to prevent forwarding any DNS requests from internal networks. As a workaround, run containers intended to be solely attached to internal networks with a custom upstream address, which will force all upstream DNS queries to be resolved from the container's network namespace.


CVE-2023-49785: NextChat, also known as ChatGPT-Next-Web, is a cross-platform chat user interface for use with ChatGPT. Versions 2.11.2

Summary: NextChat (also called ChatGPT-Next-Web) version 2.11.2 and earlier has two security flaws: SSRF (server-side request forgery, where attackers trick the server into making unwanted requests) and XSS (cross-site scripting, where attackers inject malicious code into web pages). These flaws let attackers read internal server data, make changes to it, hide their location by routing traffic through the app, or attack other targets on the internet.

Solution: According to the source: "Users may avoid exposing the application to the public internet or, if exposing the application to the internet, ensure it is an isolated network with no access to any other internal resources." The source also notes that as of publication, no patch is available.

OpenAI ChatGPT NextChat ChatGPT-Next-Web


CVE-2024-27565: A Server-Side Request Forgery (SSRF) in weixin.php of ChatGPT-wechat-personal commit a0857f6 allows attackers to force t

Summary: CVE-2024-27565 is a server-side request forgery (SSRF, a flaw that allows attackers to trick a server into making unwanted requests to other systems) vulnerability found in the weixin.php file of ChatGPT-wechat-personal at commit a0857f6. This vulnerability lets attackers force the application to make arbitrary requests on their behalf. The vulnerability has a CVSS 4.0 severity rating (a moderate score on a 0-10 scale measuring how serious a security flaw is).

Solution: N/A -- no mitigation discussed in source.


CVE-2024-28088: LangChain through 0.1.10 allows ../ directory traversal by an actor who is able to control the final part of the path pa

Summary: LangChain versions up to 0.1.10 have a path traversal vulnerability (a flaw where an attacker can use ../ sequences to access files outside the intended directory) that allows someone controlling part of a file path to load configurations from anywhere instead of just the intended GitHub repository, potentially exposing API keys or enabling remote code execution (running malicious commands on a system). This bug affects how the load_chain function handles file paths.

Solution: A patch is available in langchain-core version 0.1.29 and later. Update to this version or newer to fix the vulnerability.


Who Am I? Conditional Prompt Injection Attacks with Microsoft Copilot

Summary: Attackers can create conditional prompt injection attacks (tricking an AI by hiding malicious instructions in its input that activate only for specific users) against Microsoft Copilot by leveraging user identity information like names and job titles that the AI includes in its context. A researcher demonstrated this by sending an email with hidden instructions that made Copilot behave differently depending on which person opened it, showing that LLM applications become more vulnerable as attackers learn to target specific users rather than all users equally.

Microsoft
Microsoft 365 Copilot
Microsoft Defender
Copilot


CVE-2024-2057: A vulnerability was found in LangChain langchain_community 0.0.26. It has been classified as critical. Affected is the f

Summary: A critical vulnerability was found in LangChain's langchain_community library version 0.0.26 in the TFIDFRetriever component (a tool that retrieves relevant documents for AI systems). The flaw allows server-side request forgery (SSRF, where an attacker tricks a server into making unwanted network requests on their behalf), and it can be exploited remotely.

Solution: Upgrading to version 0.0.27 addresses this issue.


AI Act Implementation: Timelines & Next steps

Summary: The EU AI Act is a regulatory framework that requires companies to comply with rules for different types of AI systems on specific timelines, starting with prohibitions on the riskiest AI uses within 6 months and expanding to cover high-risk AI systems (such as those used in law enforcement, hiring, or education) by 24 months after the law takes effect. The article outlines key compliance deadlines, secondary laws the EU Commission might create to clarify the rules, and guidance documents to help organizations understand how to follow the AI Act.


CVE-2024-25723: ZenML Server in the ZenML machine learning package before 0.46.7 for Python allows remote privilege escalation because t

Summary: ZenML Server in the ZenML machine learning package before version 0.46.7 has a remote privilege escalation vulnerability (CVE-2024-25723), meaning an attacker can gain higher-level access to the system from a distance. The flaw exists in a REST API endpoint (a web-based interface for requests) that activates user accounts, because it only requires a valid username and new password to change account settings, without proper access controls checking who should be allowed to do this.

Solution: Update ZenML to version 0.46.7 or use one of the patched versions: 0.44.4, 0.43.1, or 0.42.2.


High-level summary of the AI Act

Summary: The EU AI Act classifies AI systems by risk level, from prohibited (like social scoring systems that manipulate behavior) to minimal risk (unregulated). High-risk AI systems, such as those used in critical decisions affecting people's lives, face strict regulations requiring developers to provide documentation, conduct testing, and monitor for problems. General-purpose AI (large language models that can do many tasks) have lighter requirements unless they present systemic risk, in which case developers must test them against adversarial attacks (attempts to trick or break them) and report serious incidents.


CVE-2024-27444: langchain_experimental (aka LangChain Experimental) in LangChain before 0.1.8 allows an attacker to bypass the CVE-2023-

Summary: CVE-2024-27444 is a vulnerability in LangChain Experimental (a Python library for building AI applications) before version 0.1.8 that allows attackers to bypass a previous security fix and run arbitrary code (malicious commands they choose) by using Python's special attributes like __import__ and __globals__, which were not blocked by the pal_chain/base.py security checks.

Solution: Update to LangChain version 0.1.8 or later. A patch is available at https://github.com/langchain-ai/langchain/commit/de9a6cdf163ed00adaf2e559203ed0a9ca2f1de7.


CVE-2024-27133: Insufficient sanitization in MLflow leads to XSS when running a recipe that uses an untrusted dataset. This issue leads

Summary: MLflow, a machine learning platform, has a vulnerability where it doesn't properly clean user input from dataset tables, allowing XSS (cross-site scripting, where attackers inject malicious code into web pages). When someone runs a recipe using an untrusted dataset in Jupyter Notebook, this can lead to RCE (remote code execution, where an attacker can run commands on the user's computer).

Solution: A patch is available at https://github.com/mlflow/mlflow/pull/10893


CVE-2024-27132: Insufficient sanitization in MLflow leads to XSS when running an untrusted recipe.

This issue leads to a client-side RC

Summary: MLflow has a vulnerability (CVE-2024-27132) where template variables are not properly sanitized, allowing XSS (cross-site scripting, where malicious code runs in a user's browser) when running an untrusted recipe in Jupyter Notebook. This can lead to client-side RCE (remote code execution, where an attacker can run commands on the user's computer) through insufficient input cleaning.


CVE-2024-27319: Versions of the package onnx before and including 1.15.0 are vulnerable to Out-of-bounds Read as the ONNX_ASSERT and ONN

Summary: ONNX (a machine learning model format library) versions 1.15.0 and earlier have an out-of-bounds read vulnerability (accessing memory outside intended boundaries) caused by an off-by-one error in the ONNX_ASSERT and ONNX_ASSERTM functions, which handle string copying. This flaw could allow attackers to read sensitive data from memory.


CVE-2024-27318: Versions of the package onnx before and including 1.15.0 are vulnerable to Directory Traversal as the external_data fiel

Summary: ONNX (a machine learning model format) versions 1.15.0 and earlier contain a directory traversal vulnerability (a security flaw where an attacker can access files outside the intended directory) in the external_data field of tensor proto (a data structure component). This vulnerability bypasses a previous security patch, allowing attackers to potentially access files they shouldn't be able to reach.


Google Gemini: Planting Instructions For Delayed Automatic Tool Invocation

Summary: A researcher discovered a vulnerability in Google Gemini where attackers can hide instructions in emails that trick the AI into automatically calling external tools (called Extensions) without the user's knowledge. When a user asks the AI to analyze a malicious email, the AI follows the hidden instructions and invokes the tool, which is a form of request forgery (making unauthorized requests on behalf of the user).


CVE-2023-30767: Improper buffer restrictions in Intel(R) Optimization for TensorFlow before version 2.13.0 may allow an authenticated us

Summary: CVE-2023-30767 is a vulnerability in Intel's Optimization for TensorFlow before version 2.13.0 caused by improper buffer restrictions (inadequate checks on how much data can be written to a memory area). An authenticated user with local access to a system could exploit this flaw to gain higher privilege levels than they should have.

Solution: Update Intel Optimization for TensorFlow to version 2.13.0 or later.

Intel
Intel Optimization for TensorFlow
TensorFlow


ChatGPT: Lack of Isolation between Code Interpreter sessions of GPTs

Summary: ChatGPT's Code Interpreter (a sandbox environment that runs code) was not properly isolated between different GPTs, meaning files uploaded to one GPT were visible and could be modified by other GPTs used by the same person, creating a security risk where malicious GPTs could steal or overwrite sensitive files. OpenAI addressed this vulnerability in May 2024.

Solution: OpenAI addressed this vulnerability in May 2024. Additionally, the source recommends: 'Disable Code Interpreter in private GPTs with private knowledge files (as they will be accessible to other GPTs)' and notes that 'when creating a new GPT Code Interpreter is off by default' as one change OpenAI made. Users should avoid uploading sensitive files to Code Interpreter and use third-party GPTs with caution, especially those with Code Interpreter enabled.


Video: ASCII Smuggling and Hidden Prompt Instructions

Summary: Researchers discovered ASCII Smuggling, a technique using Unicode Tags Block characters (special Unicode codes that mirror ASCII but stay invisible in UI elements) to hide prompt injections (tricky instructions hidden in AI input) that large language models interpret as regular text. This attack is particularly dangerous for LLMs because they can both read these hidden messages and generate them in responses, enabling more sophisticated attacks beyond traditional methods like XSS (cross-site scripting, injecting malicious code into websites) and SSRF (server-side request forgery, tricking a server into making unauthorized requests).

Solution: As a developer, a possible mitigation is to remove Unicode Tags Block text on the way in and out (meaning filter it both when users send input to your LLM and when the LLM sends responses back to users). Additionally, test your own LLM applications for this new attack vector to identify vulnerabilities.


Hidden Prompt Injections with Anthropic Claude

Summary: A researcher discovered that Anthropic's Claude AI model is vulnerable to hidden prompt injections using Unicode Tags code points (invisible characters that can carry secret instructions in text). Like ChatGPT before it, Claude can interpret these hidden instructions and follow them, even though users cannot see them on their screen. The researcher reported the issue to Anthropic, but the ticket was closed without further details provided.


CVE-2024-0964: A local file include could be remotely triggered in Gradio due to a vulnerable user-supplied JSON value in an API reques

Summary: CVE-2024-0964 is a vulnerability in Gradio (an AI tool library) where an attacker can remotely read files from a server by sending a specially crafted JSON request. The flaw exists because Gradio doesn't properly limit which files users can access through its API, allowing attackers to bypass directory restrictions and read sensitive files they shouldn't be able to reach.

Solution: A patch is available at https://github.com/gradio-app/gradio/commit/d76bcaaaf0734aaf49a680f94ea9d4d22a602e70, which addresses the path traversal vulnerability (CWE-22, improper limitation of pathname access).


Exploring Google Bard's Data Visualization Feature (Code Interpreter)

Summary: Google Bard gained a code interpreter feature that lets it run Python code to create charts and perform calculations. The feature works by executing code in a sandboxed environment (an isolated virtual computer), which users can trigger by asking Bard to visualize data or plot results. While exploring this sandbox, the author found it to be somewhat unreliable and less capable than similar features in other AI systems, with limited ability to run arbitrary programs.


CVE-2024-23751: LlamaIndex (aka llama_index) through 0.9.34 allows SQL injection via the Text-to-SQL feature in NLSQLTableQueryEngine, S

Summary: LlamaIndex (a tool for building AI applications with custom data) versions up to 0.9.34 has a SQL injection vulnerability (a flaw where attackers can insert malicious database commands into normal text input) in its Text-to-SQL feature. This allows attackers to run harmful SQL commands by hiding them in English language requests, such as deleting database tables.


CVE-2024-23730: The OpenAPI and ChatGPT plugin loaders in LlamaHub (aka llama-hub) before 0.0.67 allow attackers to execute arbitrary co

Summary: LlamaHub (a library for loading plugins) versions before 0.0.67 have a vulnerability in how they handle OpenAPI and ChatGPT plugin loaders that allows attackers to execute arbitrary code (run any code they choose on a system). The problem is that the code uses unsafe YAML parsing instead of safe_load (a secure function that prevents malicious code in configuration files).

Solution: Upgrade LlamaHub to version 0.0.67 or later, as indicated by the release notes and patch references in the source.


AWS Fixes Data Exfiltration Attack Angle in Amazon Q for Business

Summary: A researcher discovered that Amazon Q for Business was vulnerable to an indirect prompt injection attack (a technique where an attacker hides malicious instructions in data that gets fed to an AI), which could trick the AI into outputting markdown tags that render as hyperlinks. This allowed attackers to steal sensitive data from victims by embedding malicious links in uploaded files. Amazon identified and fixed the vulnerability after the researcher reported it.


ASCII Smuggler Tool: Crafting Invisible Text and Decoding Hidden Codes

Summary: A researcher discovered that LLMs like ChatGPT can be tricked through prompt injection (hiding malicious instructions in input text) by using invisible Unicode characters from the Tags Unicode Block (a section of the Unicode standard containing special code points). The proof-of-concept demonstrated how invisible instructions embedded in pasted text caused ChatGPT to perform unintended actions, such as generating images with DALL-E.


CVE-2023-31036: NVIDIA Triton Inference Server for Linux and Windows contains a vulnerability where, when it is launched with the non-de

Summary: NVIDIA Triton Inference Server for Linux and Windows has a vulnerability (CVE-2023-31036) that occurs when launched with the non-default --model-control explicit option, allowing attackers to use path traversal (exploiting how file paths are handled to access unintended directories) through the model load API. A successful attack could lead to code execution (running unauthorized commands), denial of service (making the system unavailable), privilege escalation (gaining higher access levels), information disclosure (exposing sensitive data), and data tampering (modifying files).


CVE-2023-7215: A vulnerability, which was classified as problematic, has been found in Chanzhaoyu chatgpt-web 2.11.1. This issue affect

Summary: CVE-2023-7215 is a cross-site scripting (XSS) vulnerability, a type of attack where malicious code gets injected into a webpage that a user views in their browser, found in Chanzhaoyu chatgpt-web version 2.11.1. An attacker can exploit this by manipulating the Description argument with malicious image code, and the attack can be performed remotely over the internet. The vulnerability has been publicly disclosed and may already be in use by attackers.


37th Chaos Communication Congress: New Important Instructions (Video + Slides)

Summary: A security researcher presented at the 37th Chaos Communication Congress about Large Language Models Application Security and prompt injection (tricking an AI by hiding instructions in its input). The talk covered security research findings and was made available in video and slide formats for public access.


CVE-2023-51449: Gradio is an open-source Python package that allows you to quickly build a demo or web application for your machine lear

Summary: Gradio is a Python package for building web demos of machine learning models. Versions before 4.11.0 had a file traversal vulnerability (a weakness that lets attackers read files they shouldn't access) in the `/file` route, allowing attackers to view arbitrary files on machines running publicly accessible Gradio apps if they knew the file paths.

Solution: Update Gradio to version 4.11.0 or later, where this issue has been patched.


CVE-2023-7018: Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.

Summary: CVE-2023-7018 is a deserialization of untrusted data vulnerability (a flaw where an AI library unsafely processes data from untrusted sources) in the Hugging Face Transformers library before version 4.36. This weakness could potentially allow an attacker to execute malicious code through specially crafted input.

Solution: Update to Transformers version 4.36 or later. A patch is available at the GitHub commit: https://github.com/huggingface/transformers/commit/1d63b0ec361e7a38f1339385e8a5a855085532ce


OpenAI Begins Tackling ChatGPT Data Leak Vulnerability

Summary: OpenAI has begun addressing a data exfiltration vulnerability (where attackers steal user data) in ChatGPT that exploits image markdown rendering during prompt injection attacks (tricking an AI by hiding instructions in its input). The company implemented a client-side validation check called 'url_safe' on the web app that blocks images from suspicious domains, though the fix is incomplete and attackers can still leak small amounts of data through workarounds.

Solution: OpenAI implemented a mitigation by adding a client-side validation API call (url_safe endpoint) that checks whether image URLs are safe before rendering them. The validation returns {"safe":false} to prevent rendering images from malicious domains. However, the source explicitly notes this is not a complete fix and suggests OpenAI should additionally "limit the number of images that are rendered per response to just one or maybe a handful maximum" to further reduce bypass techniques. The source also notes the current iOS version 1.2023.347 (16603) does not yet have these improvements.


CVE-2023-6730: Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.

Summary: CVE-2023-6730 is a deserialization of untrusted data vulnerability (a security flaw where a program unsafely reconstructs objects from untrusted input, potentially allowing attackers to execute malicious code) found in the Hugging Face Transformers library before version 4.36. The vulnerability has a CVSS score of 4.0, which indicates a moderate severity level (a 0-10 rating of how severe a vulnerability is).


CVE-2023-6909: Path Traversal: '\..\filename' in GitHub repository mlflow/mlflow prior to 2.9.2.

Summary: CVE-2023-6909 is a path traversal vulnerability (a security flaw where an attacker can access files outside their intended directory using special characters like '..\'). It affects MLflow versions before 2.9.2 in the mlflow/mlflow GitHub repository. The vulnerability was discovered and reported through the huntr.dev bug bounty platform.

Solution: Update MLflow to version 2.9.2 or later. A patch is available at the GitHub commit referenced: https://github.com/mlflow/mlflow/commit/1da75dfcecd4d169e34809ade55748384e8af6c1


CVE-2023-6831: Path Traversal: '\..\filename' in GitHub repository mlflow/mlflow prior to 2.9.2.

Summary: CVE-2023-6831 is a path traversal vulnerability (a flaw where an attacker can access files outside the intended directory by using special characters like '..\'). in MLflow versions before 2.9.2 that allows attackers to manipulate file paths and access restricted files they shouldn't be able to reach.

Solution: Update MLflow to version 2.9.2 or later. A patch is available at https://github.com/mlflow/mlflow/commit/1da75dfcecd4d169e34809ade55748384e8af6c1.


CVE-2023-6572: Command Injection in GitHub repository gradio-app/gradio prior to main.

Summary: CVE-2023-6572 is a command injection vulnerability (a security flaw where an attacker can run unauthorized commands) in the Gradio application (a tool for building AI demos) versions prior to the main branch. The vulnerability results from improper handling of special characters that could allow attackers to execute commands on affected systems.

Solution: A patch is available at the GitHub commit: https://github.com/gradio-app/gradio/commit/5b5af1899dd98d63e1f9b48a93601c2db1f56520. Users should update to the main branch or apply this commit to fix the vulnerability.


CVE-2023-6753: Path Traversal in GitHub repository mlflow/mlflow prior to 2.9.2.

Summary: CVE-2023-6753 is a path traversal vulnerability (a security flaw where an attacker can access files outside the intended directory by using special path characters) found in MLflow versions before 2.9.2. The vulnerability allows unauthorized access to restricted files on a system running the affected software.

Solution: Update MLflow to version 2.9.2 or later. A patch is available at https://github.com/mlflow/mlflow/commit/1c6309f884798fbf56017a3cc808016869ee8de4.


Malicious ChatGPT Agents: How GPTs Can Quietly Grab Your Data (Demo)

Summary: A researcher demonstrated that malicious GPTs (custom ChatGPT agents) can secretly steal user data by embedding hidden images in conversations that send information to external servers, and can also trick users into sharing personal details like passwords. OpenAI's validation checks for publishing GPTs can be easily bypassed by slightly rewording malicious instructions, allowing harmful GPTs to be shared publicly, though the researcher reported these vulnerabilities to OpenAI in November 2023 without receiving a fix.

Solution: N/A -- no mitigation discussed in source.

OpenAI
ChatGPT
GPTs
Bing Chat
Google Bard
Anthropic Claude


CVE-2023-35625: Azure Machine Learning Compute Instance for SDK Users Information Disclosure Vulnerability

Summary: CVE-2023-35625 is a vulnerability in Azure Machine Learning Compute Instance that allows unauthorized users to access sensitive information through the SDK (software development kit, a collection of tools for building applications). The vulnerability is classified as an information disclosure issue, meaning private data could be exposed to people who shouldn't see it.


CVE-2023-6709: Improper Neutralization of Special Elements Used in a Template Engine in GitHub repository mlflow/mlflow prior to 2.9.2.

Summary: CVE-2023-6709 is a vulnerability in MLflow (a machine learning tool) versions before 2.9.2 involving improper neutralization of special elements in a template engine (a system that generates text by filling in placeholders in templates). This weakness could potentially allow attackers to manipulate how the software processes certain input data.

Solution: Update MLflow to version 2.9.2 or later. A patch is available at https://github.com/mlflow/mlflow/commit/432b8ccf27fd3a76df4ba79bb1bec62118a85625.


CVE-2023-6568: A reflected Cross-Site Scripting (XSS) vulnerability exists in the mlflow/mlflow repository, specifically within the han

Summary: MLflow, an open-source machine learning platform, has a reflected XSS (cross-site scripting, where an attacker injects malicious JavaScript that runs in a victim's browser) vulnerability in how it handles the Content-Type header in POST requests. An attacker can craft a malicious Content-Type header that gets sent back to the user without proper filtering, allowing arbitrary JavaScript code to execute in the victim's browser.


CVE-2023-43472: An issue in MLFlow versions 2.8.1 and before allows a remote attacker to obtain sensitive information via a crafted requ

Summary: CVE-2023-43472 is a vulnerability in MLFlow (an open-source platform for managing machine learning workflows) versions 2.8.1 and earlier that allows a remote attacker to obtain sensitive information by sending a specially crafted request to the REST API (the interface that programs use to communicate with MLFlow). The vulnerability has a CVSS severity score of 4.0 (a moderate risk level on a scale of 0-10).


Ekoparty Talk - Prompt Injections in the Wild

Summary: A security researcher presented at Ekoparty 2023 about prompt injections (attacks where malicious instructions are hidden in inputs to trick an AI into misbehaving) found in real-world LLM applications and chatbots like ChatGPT, Bing Chat, and Google Bard, demonstrating various exploits and discussing mitigations. The talk covered both basic LLM concepts and deep dives into how these attacks work across different AI platforms.

OpenAI Anthropic Google Microsoft Amazon

Bing Chat
ChatGPT
Anthropic Claude
Azure AI
GCP Vertex AI
Google Bard


CVE-2023-48299: TorchServe is a tool for serving and scaling PyTorch models in production. Starting in version 0.1.0 and prior to versio

Summary: TorchServe (a tool for running PyTorch machine learning models as web services) versions before 0.9.0 had a ZipSlip vulnerability (a flaw where an attacker can extract files outside the intended folder by crafting malicious archive files), allowing attackers to upload harmful code disguised in publicly available models that could execute on machines running TorchServe. The vulnerability affected the model and workflow management API, which handles uploaded files.

Solution: Upgrade to TorchServe version 0.9.0 or later. The fix validates the file paths in zip archives before extracting them to prevent files from being placed in unintended filesystem locations.


CVE-2023-46302: Apache Software Foundation Apache Submarine has a bug when serializing against yaml. The bug is caused by snakeyaml  htt

Summary: Apache Submarine has a security vulnerability in how it handles YAML (a data format language) requests because it uses an unsafe library called snakeyaml. When users send YAML data to the application through its REST API (a system for receiving web requests), the unsafe handling could allow attackers to execute malicious code.

Solution: Users should upgrade to Apache Submarine version 0.8.0, which fixes this issue by replacing snakeyaml with jackson-dataformat-yaml. If upgrading is not possible, users can cherry-pick (apply a specific code fix from) PR https://github.com/apache/submarine/pull/1054 and rebuild the submarine-server image.