CVE-2024-34072: sagemaker-python-sdk is a library for training and deploying machine learning models on Amazon SageMaker. The sagemaker.

Summary: A vulnerability in the sagemaker-python-sdk library (used for machine learning on Amazon SageMaker) allows unsafe deserialization, where the NumpyDeserializer module can execute malicious code if it processes untrusted pickled data (serialized Python objects stored in a binary format). An attacker could exploit this to run arbitrary commands on a system or crash it.

Solution: Upgrade to sagemaker-python-sdk version 2.218.0 or later. If unable to upgrade, do not process pickled numpy object arrays from untrusted sources or data that could have been modified by others. Only use pickled numpy object arrays from sources you trust.


CVE-2023-5675: A flaw was found in Quarkus. When a Quarkus RestEasy Classic or Reactive JAX-RS endpoint has its methods declared in the

Summary: CVE-2023-5675 is a security flaw in Quarkus (a Java framework for building applications) where authorization checks are bypassed for REST API endpoints whose methods are defined in abstract classes or modified by extensions using annotation processors, if certain security settings are enabled. This means unauthorized users could potentially access protected API endpoints that should require authentication or specific permissions.

Solution: N/A -- no mitigation discussed in source.


CVE-2024-31584: Pytorch before v2.2.0 has an Out-of-bounds Read vulnerability via the component torch/csrc/jit/mobile/flatbuffer_loader.

Summary: PyTorch versions before 2.2.0 contain an out-of-bounds read vulnerability (a bug where code tries to read data from memory outside its allowed range) in the flatbuffer_loader component, which is used for loading machine learning models on mobile devices. This vulnerability could potentially allow attackers to read sensitive information from memory or cause the program to crash.

Solution: Upgrade to PyTorch version 2.2.0 or later. A patch is available at https://github.com/pytorch/pytorch/commit/7c35874ad664e74c8e4252d67521f3986eadb0e6.


CVE-2024-31583: Pytorch before version v2.2.0 was discovered to contain a use-after-free vulnerability in torch/csrc/jit/mobile/interpre

Summary: PyTorch versions before v2.2.0 contain a use-after-free vulnerability (a memory bug where code tries to access data that has already been freed) in the mobile interpreter component. This vulnerability was identified in the torch/csrc/jit/mobile/interpreter.cpp file.

Solution: Update PyTorch to version v2.2.0 or later. A patch is available at https://github.com/pytorch/pytorch/commit/9c7071b0e324f9fb68ab881283d6b8d388a4bcd2.


CVE-2024-31580: PyTorch before v2.2.0 was discovered to contain a heap buffer overflow vulnerability in the component /runtime/vararg_fu

Summary: PyTorch versions before v2.2.0 contain a heap buffer overflow vulnerability (a type of memory safety bug where a program writes data beyond allocated memory limits) in its runtime component that allows attackers to crash the software through specially crafted input. This is a Denial of Service attack, meaning the goal is to make the software unusable rather than steal data.

Solution: Upgrade to PyTorch v2.2.0 or later. A patch is available at https://github.com/pytorch/pytorch/commit/b5c3a17c2c207ebefcb85043f0cf94be9b2fef81.


CVE-2024-3660: A arbitrary code injection vulnerability in TensorFlow's Keras framework (<2.13) allows attackers to execute arbitrary c

Summary: CVE-2024-3660 is a code injection vulnerability (a flaw that lets attackers insert and run harmful code) in TensorFlow's Keras framework (a machine learning library) affecting versions before 2.13. Attackers can exploit this to execute arbitrary code (run commands they choose) with the same permissions as the application using a vulnerable model.


CVE-2024-3573: mlflow/mlflow is vulnerable to Local File Inclusion (LFI) due to improper parsing of URIs, allowing attackers to bypass

Summary: MLflow (a machine learning platform) has a vulnerability where its URI parsing function incorrectly classifies certain file paths as non-local, allowing attackers to read sensitive files they shouldn't access. By crafting malicious model versions with specially crafted parameters, attackers can bypass security checks and read arbitrary files from the system.


CVE-2024-3571: langchain-ai/langchain is vulnerable to path traversal due to improper limitation of a pathname to a restricted director

Summary: LangChain's LocalFileStore feature has a path traversal vulnerability (a security flaw where attackers can access files outside the intended directory by using special path sequences like '../'). An attacker can exploit this to read or write any files on the system, potentially stealing data or executing malicious code. The problem stems from the mset and mget methods not properly filtering user input before handling file paths.


CVE-2024-2912: An insecure deserialization vulnerability exists in the BentoML framework, allowing remote code execution (RCE) by sendi

Summary: BentoML (a framework for building AI applications) contains an insecure deserialization vulnerability that lets attackers run arbitrary commands on servers by sending specially crafted requests. When the framework deserializes (converts stored data back into usable objects) a malicious object, it automatically executes hidden OS commands, giving attackers control of the server.


CVE-2024-1594: A path traversal vulnerability exists in the mlflow/mlflow repository, specifically within the handling of the `artifact

Summary: CVE-2024-1594 is a path traversal vulnerability (a flaw that lets attackers access files outside their permitted directory) in MLflow's experiment creation feature. Attackers can exploit this by inserting a fragment component (#) into the artifact_location parameter to read arbitrary files on the server.


CVE-2024-1593: A path traversal vulnerability exists in the mlflow/mlflow repository due to improper handling of URL parameters. By smu

Summary: MLflow, a machine learning platform, has a path traversal vulnerability (a security flaw where attackers can access files outside intended directories) caused by improper handling of URL parameters. Attackers can use the semicolon (;) character to hide malicious path sequences in URLs, potentially gaining unauthorized access to sensitive files or compromising the server.


CVE-2024-1561: An issue was discovered in gradio-app/gradio, where the `/component_server` endpoint improperly allows the invocation of

Summary: Gradio, a popular Python library for building AI interfaces, has a vulnerability in its `/component_server` endpoint that lets attackers call any method on a Component class with their own arguments. By exploiting a specific method called `move_resource_to_block_cache()`, attackers can copy files from the server's filesystem to a temporary folder and download them, potentially exposing sensitive data like API keys, especially when apps are shared online or hosted on platforms like Hugging Face.


CVE-2024-1560: A path traversal vulnerability exists in the mlflow/mlflow repository, specifically within the artifact deletion functio

Summary: A path traversal vulnerability (a security flaw where attackers use special characters like ../ to access files outside their intended directory) exists in MLflow's artifact deletion feature. Attackers can delete arbitrary files on a server by exploiting an extra decoding step that fails to properly validate user input, and this vulnerability affects versions up to 2.9.2.


CVE-2024-1558: A path traversal vulnerability exists in the `_create_model_version()` function within `server/handlers.py` of the mlflo

Summary: CVE-2024-1558 is a path traversal vulnerability (a security flaw where an attacker uses special characters like "../" to access files outside their intended directory) in MLflow's model version creation function. An attacker can craft a malicious `source` parameter that bypasses the validation check, allowing them to read any file on the server when fetching model artifacts.


CVE-2024-1483: A path traversal vulnerability exists in mlflow/mlflow version 2.9.2, allowing attackers to access arbitrary files on th

Summary: CVE-2024-1483 is a path traversal vulnerability (a weakness that lets attackers access files outside intended directories) in MLflow version 2.9.2 that allows attackers to read arbitrary files on a server. The vulnerability occurs because the server doesn't properly validate user input in the 'artifact_location' and 'source' parameters, and attackers can exploit this by sending specially crafted HTTP POST requests that use '#' instead of '?' in local URIs to navigate the server's directory structure.


CVE-2024-1183: An SSRF (Server-Side Request Forgery) vulnerability exists in the gradio-app/gradio repository, allowing attackers to sc

Summary: CVE-2024-1183 is an SSRF vulnerability (a flaw where an attacker tricks a server into making requests to internal networks) in the Gradio application that lets attackers scan and identify open ports on internal networks by manipulating the 'file' parameter in requests and reading responses for specific headers or error messages.

Solution: A patch is available at https://github.com/gradio-app/gradio/commit/2ad3d9e7ec6c8eeea59774265b44f11df7394bb4


Bobby Tables but with LLM Apps - Google NotebookLM Data Exfiltration

Summary: Google's NotebookLM is a tool that lets users upload files for an AI to analyze, but it's vulnerable to prompt injection (tricking the AI by hiding instructions in uploaded files) that can manipulate the AI's responses and expose what users see. The tool also has a data exfiltration vulnerability (attackers stealing information) when processing untrusted files, and there is currently no known way to prevent these attacks, meaning users cannot fully trust the AI's responses when working with files from unknown sources.


CVE-2024-31462: stable-diffusion-webui is a web interface for Stable Diffusion, implemented using Gradio library. Stable-diffusion-webui

Summary: Stable-diffusion-webui version 1.7.0 has a vulnerability where user input from the Backup/Restore tab is not properly validated before being used to create file paths, allowing attackers to write JSON files to arbitrary locations on Windows systems where the web server has access. This is a limited file write vulnerability (a security flaw that lets attackers create or modify files in unintended locations) that could let an attacker place malicious files on the server.

Stability AI
stable-diffusion-webui
Gradio


CVE-2023-51409: Unrestricted Upload of File with Dangerous Type vulnerability in Jordy Meow AI Engine: ChatGPT Chatbot.This issue affect

Summary: CVE-2023-51409 is a vulnerability in the Jordy Meow AI Engine: ChatGPT Chatbot plugin (versions up to 1.9.98) that allows unrestricted upload of dangerous file types, meaning attackers can upload files that shouldn't be allowed without proper validation. This vulnerability could potentially lead to remote code execution (running malicious commands on the affected system).

Jordy Meow AI Engine
ChatGPT Chatbot plugin


CVE-2024-3568: The huggingface/transformers library is vulnerable to arbitrary code execution through deserialization of untrusted data

Summary: The huggingface/transformers library has a vulnerability where attackers can run arbitrary code on a victim's machine by tricking them into loading a malicious checkpoint file. The problem occurs in the `load_repo_checkpoint()` function, which uses `pickle.load()` (a Python function that reconstructs objects from serialized data) on data that might come from untrusted sources, allowing attackers to execute remote code execution (RCE, where an attacker runs commands on a system they don't own).


CVE-2024-2221: qdrant/qdrant is vulnerable to a path traversal and arbitrary file upload vulnerability via the `/collections/{COLLECTIO

Summary: Qdrant (a vector database software) has a vulnerability in its snapshot upload endpoint that allows attackers to upload files to any location on the server's filesystem through path traversal (using special file path sequences to access directories they shouldn't). This could let attackers execute arbitrary code on the server and damage the system's integrity and availability.

Solution: A patch is available at https://github.com/qdrant/qdrant/commit/e6411907f0ecf3c2f8ba44ab704b9e4597d9705d


CVE-2024-1728: gradio-app/gradio is vulnerable to a local file inclusion vulnerability due to improper validation of user-supplied inpu

Summary: Gradio (a framework for building AI interfaces) has a vulnerability in its UploadButton component where it doesn't properly validate (check) user input, allowing attackers to read any file on the server by manipulating file paths sent to the `/queue/join` endpoint. This could let attackers steal sensitive files like SSH keys (credentials used for secure server access) and potentially execute arbitrary code on the system.


CVE-2024-3098: A vulnerability was identified in the `exec_utils` class of the `llama_index` package, specifically within the `safe_eva

Summary: A vulnerability was found in the `safe_eval` function of the `llama_index` package that allows prompt injection (tricking an AI by hiding instructions in its input) to execute arbitrary code (running code an attacker chooses). The flaw exists because the input validation is insufficient, meaning the package doesn't properly check what data is being passed in, allowing attackers to bypass safety restrictions that were meant to prevent this type of attack.


CVE-2024-28224: Ollama before 0.1.29 has a DNS rebinding vulnerability that can inadvertently allow remote access to the full API, there

Summary: Ollama before version 0.1.29 has a DNS rebinding vulnerability (a technique where an attacker tricks a system into connecting to a malicious server by manipulating how domain names are translated into addresses), which allows unauthorized remote access to its full API. This vulnerability could let an attacker interact with the language model, remove models, or cause a denial of service (making a system unavailable by overloading it with requests).

Solution: Update Ollama to version 0.1.29 or later.

confidentiality integrity availability safety


CVE-2024-31224: GPT Academic provides interactive interfaces for large language models. A vulnerability was found in gpt_academic versio

Summary: GPT Academic is a tool that provides interactive interfaces for large language models. Versions 3.64 through 3.73 have a vulnerability where the server deserializes untrusted data (processes data from users without verifying it's safe), which could allow attackers to execute code remotely on any exposed server. Any device running these vulnerable versions and accessible over the internet is at risk.

Solution: Upgrade to version 3.74, which contains a patch for the issue. The source states: 'There are no known workarounds aside from upgrading to a patched version.'


Google AI Studio Data Exfiltration via Prompt Injection - Possible Regression and Fix

Summary: Google AI Studio had a vulnerability that allowed attackers to steal data through prompt injection (tricking an AI by hiding malicious instructions in its input), where a malicious file could trick the AI into exfiltrating other uploaded files to an attacker's server via image tags. The vulnerability appeared in a recent update but was fixed within 12 days of being reported to Google on February 17, 2024.

Solution: The issue was fixed by Google and did not reproduce after the company heard back about the report 12 days later (by approximately February 29, 2024). The ticket was closed as 'Duplicate' on March 3, 2024, suggesting the vulnerability may have also been caught through internal testing.


The dangers of AI agents unfurling hyperlinks and what to do about it

Summary: Unfurling is when an application automatically expands hyperlinks to show previews, which can be exploited in AI chatbots to leak data. When an attacker uses prompt injection (tricking an AI by hiding instructions in its input) to make the chatbot generate a link containing sensitive information from earlier conversations, the unfurling feature automatically sends that data to a third-party server, potentially exposing private information.

Solution: To disable unfurling in Slack Apps, modify the message creation function to include unfurl settings in the JSON object: set "unfurl_links": False and "unfurl_media": False when creating the message, as shown in the example code: def create_message(text): message = { "text": text, "unfurl_links": False, "unfurl_media": False } return json.dumps(message)


CVE-2024-3078: A vulnerability was found in Qdrant up to 1.6.1/1.7.4/1.8.2 and classified as critical. This issue affects some unknown

Summary: A critical vulnerability was discovered in Qdrant (a vector database system) versions up to 1.6.1, 1.7.4, and 1.8.2 that allows path traversal (a technique where attackers access files outside intended directories) through the Full Snapshot REST API (a web interface for creating system backups). This flaw could let attackers manipulate file paths to access unauthorized files on the system.

Solution: Upgrade to Qdrant version 1.8.3 or later. The specific patch is identified as 3ab5172e9c8f14fa1f7b24e7147eac74e2412b62.


CVE-2024-1729: A timing attack vulnerability exists in the gradio-app/gradio repository, specifically within the login function in rout

Summary: CVE-2024-1729 is a timing attack vulnerability (where an attacker guesses a password by measuring how long the system takes to reject it) in the Gradio application's login function. The vulnerability exists because the code directly compares the entered password with the stored password using a simple equality check, which can leak information through response time differences, potentially allowing attackers to bypass authentication and gain unauthorized access.

Solution: A patch is available at https://github.com/gradio-app/gradio/commit/e329f1fd38935213fe0e73962e8cbd5d3af6e87b. Additionally, a bounty reference with more details is provided at https://huntr.com/bounties/f6a10a8d-f538-4cb7-9bb2-85d9f5708124.


CVE-2024-29100: Unrestricted Upload of File with Dangerous Type vulnerability in Jordy Meow AI Engine: ChatGPT Chatbot.This issue affect

Summary: CVE-2024-29100 is an unrestricted file upload vulnerability (a security flaw that allows attackers to upload harmful files without proper checks) in the Jordy Meow AI Engine: ChatGPT Chatbot plugin for WordPress, affecting versions up to 2.1.4. This vulnerability could potentially allow attackers to upload dangerous files to a website using this plugin.


CVE-2024-29090: Server-Side Request Forgery (SSRF) vulnerability in Jordy Meow AI Engine: ChatGPT Chatbot.This issue affects AI Engine:

Summary: A server-side request forgery (SSRF, a vulnerability where an attacker tricks a server into making unintended requests to other systems) vulnerability was found in the AI Engine: ChatGPT Chatbot plugin by Jordy Meow, affecting versions up to 2.1.4. The vulnerability allows authenticated attackers to exploit the plugin to perform unauthorized requests.


CVE-2024-1540: A command injection vulnerability exists in the deploy+test-visual.yml workflow of the gradio-app/gradio repository, due

Summary: CVE-2024-1540 is a command injection vulnerability (a weakness where an attacker can insert malicious commands into code that gets executed) in the gradio-app/gradio repository's workflow file. Attackers could exploit this by manipulating GitHub context information within expressions to run unauthorized commands, potentially stealing secrets or modifying the repository. The vulnerability stems from unsafe handling of variables that are directly substituted into scripts before execution.

Solution: Remediation involves setting untrusted input values to intermediate environment variables to prevent direct influence on script generation.


CVE-2024-2206: An SSRF vulnerability exists in the gradio-app/gradio due to insufficient validation of user-supplied URLs in the `/prox

Summary: CVE-2024-2206 is an SSRF vulnerability (server-side request forgery, where an attacker tricks a server into making requests to unintended targets) in Gradio, an AI framework. Attackers can exploit this by sending specially crafted requests with an `X-Direct-Url` header to add arbitrary URLs to a list that the application uses for proxying (forwarding) requests, potentially allowing unauthorized access to internal systems. The vulnerability exists because the application does not properly validate URLs in its `build_proxy_request` function.


CVE-2024-1455: A vulnerability in the langchain-ai/langchain repository allows for a Billion Laughs Attack, a type of XML External Enti

Summary: CVE-2024-1455 is a vulnerability in the langchain-ai/langchain repository that allows a Billion Laughs Attack, a type of XML External Entity (XXE) exploitation where an attacker nests multiple layers of entities within an XML document to make the parser consume excessive CPU and memory resources, causing a denial of service (DoS, where a system becomes unavailable to legitimate users).

Solution: A patch is available at https://github.com/langchain-ai/langchain/commit/727d5023ce88e18e3074ef620a98137d26ff92a3


The AI Office is hiring

Summary: The European Commission is hiring AI specialists to work in the AI Office, which will enforce the EU's AI Act by overseeing compliance of general-purpose AI models (large AI systems available to the public). The office will have real regulatory powers to require companies to implement safety measures, restrict models, or remove them from the market, and will develop evaluation tools and benchmarks to identify dangerous AI behaviors.


CVE-2024-1727: A Cross-Site Request Forgery (CSRF) vulnerability in gradio-app/gradio allows attackers to upload multiple large files t

Summary: CVE-2024-1727 is a CSRF vulnerability (cross-site request forgery, where an attacker tricks a victim into making unintended requests) in Gradio that lets attackers upload large files to a victim's computer without permission. An attacker can create a malicious webpage that, when visited, automatically uploads files to the victim's system, potentially filling up their disk space and causing a denial of service (making the system unusable).

Solution: A patch is available at https://github.com/gradio-app/gradio/commit/84802ee6a4806c25287344dce581f9548a99834a


The AI Office: What is it, and how does it work?

Summary: The European AI Office is a new EU regulator created to oversee general purpose AI (GPAI) models and systems, which are AI systems designed to perform a wide range of tasks, across all 27 EU Member States under the AI Act. It monitors compliance, analyzes emerging risks, develops evaluation capabilities, produces voluntary codes of practice for companies to follow, and coordinates enforcement between national regulators and international partners. The Office also supports small and medium businesses with compliance resources and oversees regulatory sandboxes, which are controlled environments where companies can test AI systems before full deployment.


CVE-2024-29037: datahub-helm provides the Kubernetes Helm charts for deploying Datahub and its dependencies on a Kubernetes cluster. Sta

Summary: A vulnerability in datahub-helm (Helm charts, which are templates for deploying applications on Kubernetes clusters) versions 0.1.143 through 0.2.181 allowed personal access tokens (credentials that grant access to the system) to be created using a publicly known default secret key instead of a random one. This meant attackers could potentially generate their own valid tokens to access DataHub instances if Metadata Service Authentication (a security feature) was enabled during a specific vulnerable time period.

Solution: Update to version 0.2.182, which contains a patch for this issue. As a workaround, reset the token signing key to be a random value, which will invalidate active personal access tokens.


CVE-2024-29018: Moby is an open source container framework that is a key component of Docker Engine, Docker Desktop, and other distribut

Summary: Moby (the container framework underlying Docker) has a bug in how it handles DNS requests from internal networks (networks isolated from external communication). When a container on an internal network needs to resolve a domain name, Moby forwards the request through the host's network namespace instead of the container's own network, which can leak data to external servers that an attacker controls. Docker Desktop is not affected by this issue.

Solution: Moby releases 26.0.0, 25.0.4, and 23.0.11 are patched to prevent forwarding any DNS requests from internal networks. As a workaround, run containers intended to be solely attached to internal networks with a custom upstream address, which will force all upstream DNS queries to be resolved from the container's network namespace.


CVE-2023-49785: NextChat, also known as ChatGPT-Next-Web, is a cross-platform chat user interface for use with ChatGPT. Versions 2.11.2

Summary: NextChat (also called ChatGPT-Next-Web) version 2.11.2 and earlier has two security flaws: SSRF (server-side request forgery, where attackers trick the server into making unwanted requests) and XSS (cross-site scripting, where attackers inject malicious code into web pages). These flaws let attackers read internal server data, make changes to it, hide their location by routing traffic through the app, or attack other targets on the internet.

Solution: According to the source: "Users may avoid exposing the application to the public internet or, if exposing the application to the internet, ensure it is an isolated network with no access to any other internal resources." The source also notes that as of publication, no patch is available.

OpenAI ChatGPT NextChat ChatGPT-Next-Web


CVE-2024-27565: A Server-Side Request Forgery (SSRF) in weixin.php of ChatGPT-wechat-personal commit a0857f6 allows attackers to force t

Summary: CVE-2024-27565 is a server-side request forgery (SSRF, a flaw that allows attackers to trick a server into making unwanted requests to other systems) vulnerability found in the weixin.php file of ChatGPT-wechat-personal at commit a0857f6. This vulnerability lets attackers force the application to make arbitrary requests on their behalf. The vulnerability has a CVSS 4.0 severity rating (a moderate score on a 0-10 scale measuring how serious a security flaw is).

Solution: N/A -- no mitigation discussed in source.


CVE-2024-28088: LangChain through 0.1.10 allows ../ directory traversal by an actor who is able to control the final part of the path pa

Summary: LangChain versions up to 0.1.10 have a path traversal vulnerability (a flaw where an attacker can use ../ sequences to access files outside the intended directory) that allows someone controlling part of a file path to load configurations from anywhere instead of just the intended GitHub repository, potentially exposing API keys or enabling remote code execution (running malicious commands on a system). This bug affects how the load_chain function handles file paths.

Solution: A patch is available in langchain-core version 0.1.29 and later. Update to this version or newer to fix the vulnerability.


Who Am I? Conditional Prompt Injection Attacks with Microsoft Copilot

Summary: Attackers can create conditional prompt injection attacks (tricking an AI by hiding malicious instructions in its input that activate only for specific users) against Microsoft Copilot by leveraging user identity information like names and job titles that the AI includes in its context. A researcher demonstrated this by sending an email with hidden instructions that made Copilot behave differently depending on which person opened it, showing that LLM applications become more vulnerable as attackers learn to target specific users rather than all users equally.

Microsoft
Microsoft 365 Copilot
Microsoft Defender
Copilot


CVE-2024-2057: A vulnerability was found in LangChain langchain_community 0.0.26. It has been classified as critical. Affected is the f

Summary: A critical vulnerability was found in LangChain's langchain_community library version 0.0.26 in the TFIDFRetriever component (a tool that retrieves relevant documents for AI systems). The flaw allows server-side request forgery (SSRF, where an attacker tricks a server into making unwanted network requests on their behalf), and it can be exploited remotely.

Solution: Upgrading to version 0.0.27 addresses this issue.


AI Act Implementation: Timelines & Next steps

Summary: The EU AI Act is a regulatory framework that requires companies to comply with rules for different types of AI systems on specific timelines, starting with prohibitions on the riskiest AI uses within 6 months and expanding to cover high-risk AI systems (such as those used in law enforcement, hiring, or education) by 24 months after the law takes effect. The article outlines key compliance deadlines, secondary laws the EU Commission might create to clarify the rules, and guidance documents to help organizations understand how to follow the AI Act.


CVE-2024-25723: ZenML Server in the ZenML machine learning package before 0.46.7 for Python allows remote privilege escalation because t

Summary: ZenML Server in the ZenML machine learning package before version 0.46.7 has a remote privilege escalation vulnerability (CVE-2024-25723), meaning an attacker can gain higher-level access to the system from a distance. The flaw exists in a REST API endpoint (a web-based interface for requests) that activates user accounts, because it only requires a valid username and new password to change account settings, without proper access controls checking who should be allowed to do this.

Solution: Update ZenML to version 0.46.7 or use one of the patched versions: 0.44.4, 0.43.1, or 0.42.2.


High-level summary of the AI Act

Summary: The EU AI Act classifies AI systems by risk level, from prohibited (like social scoring systems that manipulate behavior) to minimal risk (unregulated). High-risk AI systems, such as those used in critical decisions affecting people's lives, face strict regulations requiring developers to provide documentation, conduct testing, and monitor for problems. General-purpose AI (large language models that can do many tasks) have lighter requirements unless they present systemic risk, in which case developers must test them against adversarial attacks (attempts to trick or break them) and report serious incidents.


CVE-2024-27444: langchain_experimental (aka LangChain Experimental) in LangChain before 0.1.8 allows an attacker to bypass the CVE-2023-

Summary: CVE-2024-27444 is a vulnerability in LangChain Experimental (a Python library for building AI applications) before version 0.1.8 that allows attackers to bypass a previous security fix and run arbitrary code (malicious commands they choose) by using Python's special attributes like __import__ and __globals__, which were not blocked by the pal_chain/base.py security checks.

Solution: Update to LangChain version 0.1.8 or later. A patch is available at https://github.com/langchain-ai/langchain/commit/de9a6cdf163ed00adaf2e559203ed0a9ca2f1de7.


CVE-2024-27133: Insufficient sanitization in MLflow leads to XSS when running a recipe that uses an untrusted dataset. This issue leads

Summary: MLflow, a machine learning platform, has a vulnerability where it doesn't properly clean user input from dataset tables, allowing XSS (cross-site scripting, where attackers inject malicious code into web pages). When someone runs a recipe using an untrusted dataset in Jupyter Notebook, this can lead to RCE (remote code execution, where an attacker can run commands on the user's computer).

Solution: A patch is available at https://github.com/mlflow/mlflow/pull/10893


CVE-2024-27132: Insufficient sanitization in MLflow leads to XSS when running an untrusted recipe.

This issue leads to a client-side RC

Summary: MLflow has a vulnerability (CVE-2024-27132) where template variables are not properly sanitized, allowing XSS (cross-site scripting, where malicious code runs in a user's browser) when running an untrusted recipe in Jupyter Notebook. This can lead to client-side RCE (remote code execution, where an attacker can run commands on the user's computer) through insufficient input cleaning.


CVE-2024-27319: Versions of the package onnx before and including 1.15.0 are vulnerable to Out-of-bounds Read as the ONNX_ASSERT and ONN

Summary: ONNX (a machine learning model format library) versions 1.15.0 and earlier have an out-of-bounds read vulnerability (accessing memory outside intended boundaries) caused by an off-by-one error in the ONNX_ASSERT and ONNX_ASSERTM functions, which handle string copying. This flaw could allow attackers to read sensitive data from memory.


CVE-2024-27318: Versions of the package onnx before and including 1.15.0 are vulnerable to Directory Traversal as the external_data fiel

Summary: ONNX (a machine learning model format) versions 1.15.0 and earlier contain a directory traversal vulnerability (a security flaw where an attacker can access files outside the intended directory) in the external_data field of tensor proto (a data structure component). This vulnerability bypasses a previous security patch, allowing attackers to potentially access files they shouldn't be able to reach.


Google Gemini: Planting Instructions For Delayed Automatic Tool Invocation

Summary: A researcher discovered a vulnerability in Google Gemini where attackers can hide instructions in emails that trick the AI into automatically calling external tools (called Extensions) without the user's knowledge. When a user asks the AI to analyze a malicious email, the AI follows the hidden instructions and invokes the tool, which is a form of request forgery (making unauthorized requests on behalf of the user).


CVE-2023-30767: Improper buffer restrictions in Intel(R) Optimization for TensorFlow before version 2.13.0 may allow an authenticated us

Summary: CVE-2023-30767 is a vulnerability in Intel's Optimization for TensorFlow before version 2.13.0 caused by improper buffer restrictions (inadequate checks on how much data can be written to a memory area). An authenticated user with local access to a system could exploit this flaw to gain higher privilege levels than they should have.

Solution: Update Intel Optimization for TensorFlow to version 2.13.0 or later.

Intel
Intel Optimization for TensorFlow
TensorFlow


ChatGPT: Lack of Isolation between Code Interpreter sessions of GPTs

Summary: ChatGPT's Code Interpreter (a sandbox environment that runs code) was not properly isolated between different GPTs, meaning files uploaded to one GPT were visible and could be modified by other GPTs used by the same person, creating a security risk where malicious GPTs could steal or overwrite sensitive files. OpenAI addressed this vulnerability in May 2024.

Solution: OpenAI addressed this vulnerability in May 2024. Additionally, the source recommends: 'Disable Code Interpreter in private GPTs with private knowledge files (as they will be accessible to other GPTs)' and notes that 'when creating a new GPT Code Interpreter is off by default' as one change OpenAI made. Users should avoid uploading sensitive files to Code Interpreter and use third-party GPTs with caution, especially those with Code Interpreter enabled.


Video: ASCII Smuggling and Hidden Prompt Instructions

Summary: Researchers discovered ASCII Smuggling, a technique using Unicode Tags Block characters (special Unicode codes that mirror ASCII but stay invisible in UI elements) to hide prompt injections (tricky instructions hidden in AI input) that large language models interpret as regular text. This attack is particularly dangerous for LLMs because they can both read these hidden messages and generate them in responses, enabling more sophisticated attacks beyond traditional methods like XSS (cross-site scripting, injecting malicious code into websites) and SSRF (server-side request forgery, tricking a server into making unauthorized requests).

Solution: As a developer, a possible mitigation is to remove Unicode Tags Block text on the way in and out (meaning filter it both when users send input to your LLM and when the LLM sends responses back to users). Additionally, test your own LLM applications for this new attack vector to identify vulnerabilities.


Hidden Prompt Injections with Anthropic Claude

Summary: A researcher discovered that Anthropic's Claude AI model is vulnerable to hidden prompt injections using Unicode Tags code points (invisible characters that can carry secret instructions in text). Like ChatGPT before it, Claude can interpret these hidden instructions and follow them, even though users cannot see them on their screen. The researcher reported the issue to Anthropic, but the ticket was closed without further details provided.


CVE-2024-0964: A local file include could be remotely triggered in Gradio due to a vulnerable user-supplied JSON value in an API reques

Summary: CVE-2024-0964 is a vulnerability in Gradio (an AI tool library) where an attacker can remotely read files from a server by sending a specially crafted JSON request. The flaw exists because Gradio doesn't properly limit which files users can access through its API, allowing attackers to bypass directory restrictions and read sensitive files they shouldn't be able to reach.

Solution: A patch is available at https://github.com/gradio-app/gradio/commit/d76bcaaaf0734aaf49a680f94ea9d4d22a602e70, which addresses the path traversal vulnerability (CWE-22, improper limitation of pathname access).


Exploring Google Bard's Data Visualization Feature (Code Interpreter)

Summary: Google Bard gained a code interpreter feature that lets it run Python code to create charts and perform calculations. The feature works by executing code in a sandboxed environment (an isolated virtual computer), which users can trigger by asking Bard to visualize data or plot results. While exploring this sandbox, the author found it to be somewhat unreliable and less capable than similar features in other AI systems, with limited ability to run arbitrary programs.


CVE-2024-23751: LlamaIndex (aka llama_index) through 0.9.34 allows SQL injection via the Text-to-SQL feature in NLSQLTableQueryEngine, S

Summary: LlamaIndex (a tool for building AI applications with custom data) versions up to 0.9.34 has a SQL injection vulnerability (a flaw where attackers can insert malicious database commands into normal text input) in its Text-to-SQL feature. This allows attackers to run harmful SQL commands by hiding them in English language requests, such as deleting database tables.


CVE-2024-23730: The OpenAPI and ChatGPT plugin loaders in LlamaHub (aka llama-hub) before 0.0.67 allow attackers to execute arbitrary co

Summary: LlamaHub (a library for loading plugins) versions before 0.0.67 have a vulnerability in how they handle OpenAPI and ChatGPT plugin loaders that allows attackers to execute arbitrary code (run any code they choose on a system). The problem is that the code uses unsafe YAML parsing instead of safe_load (a secure function that prevents malicious code in configuration files).

Solution: Upgrade LlamaHub to version 0.0.67 or later, as indicated by the release notes and patch references in the source.


AWS Fixes Data Exfiltration Attack Angle in Amazon Q for Business

Summary: A researcher discovered that Amazon Q for Business was vulnerable to an indirect prompt injection attack (a technique where an attacker hides malicious instructions in data that gets fed to an AI), which could trick the AI into outputting markdown tags that render as hyperlinks. This allowed attackers to steal sensitive data from victims by embedding malicious links in uploaded files. Amazon identified and fixed the vulnerability after the researcher reported it.


ASCII Smuggler Tool: Crafting Invisible Text and Decoding Hidden Codes

Summary: A researcher discovered that LLMs like ChatGPT can be tricked through prompt injection (hiding malicious instructions in input text) by using invisible Unicode characters from the Tags Unicode Block (a section of the Unicode standard containing special code points). The proof-of-concept demonstrated how invisible instructions embedded in pasted text caused ChatGPT to perform unintended actions, such as generating images with DALL-E.


CVE-2023-31036: NVIDIA Triton Inference Server for Linux and Windows contains a vulnerability where, when it is launched with the non-de

Summary: NVIDIA Triton Inference Server for Linux and Windows has a vulnerability (CVE-2023-31036) that occurs when launched with the non-default --model-control explicit option, allowing attackers to use path traversal (exploiting how file paths are handled to access unintended directories) through the model load API. A successful attack could lead to code execution (running unauthorized commands), denial of service (making the system unavailable), privilege escalation (gaining higher access levels), information disclosure (exposing sensitive data), and data tampering (modifying files).


CVE-2023-7215: A vulnerability, which was classified as problematic, has been found in Chanzhaoyu chatgpt-web 2.11.1. This issue affect

Summary: CVE-2023-7215 is a cross-site scripting (XSS) vulnerability, a type of attack where malicious code gets injected into a webpage that a user views in their browser, found in Chanzhaoyu chatgpt-web version 2.11.1. An attacker can exploit this by manipulating the Description argument with malicious image code, and the attack can be performed remotely over the internet. The vulnerability has been publicly disclosed and may already be in use by attackers.


37th Chaos Communication Congress: New Important Instructions (Video + Slides)

Summary: A security researcher presented at the 37th Chaos Communication Congress about Large Language Models Application Security and prompt injection (tricking an AI by hiding instructions in its input). The talk covered security research findings and was made available in video and slide formats for public access.


CVE-2023-51449: Gradio is an open-source Python package that allows you to quickly build a demo or web application for your machine lear

Summary: Gradio is a Python package for building web demos of machine learning models. Versions before 4.11.0 had a file traversal vulnerability (a weakness that lets attackers read files they shouldn't access) in the `/file` route, allowing attackers to view arbitrary files on machines running publicly accessible Gradio apps if they knew the file paths.

Solution: Update Gradio to version 4.11.0 or later, where this issue has been patched.


CVE-2023-7018: Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.

Summary: CVE-2023-7018 is a deserialization of untrusted data vulnerability (a flaw where an AI library unsafely processes data from untrusted sources) in the Hugging Face Transformers library before version 4.36. This weakness could potentially allow an attacker to execute malicious code through specially crafted input.

Solution: Update to Transformers version 4.36 or later. A patch is available at the GitHub commit: https://github.com/huggingface/transformers/commit/1d63b0ec361e7a38f1339385e8a5a855085532ce


OpenAI Begins Tackling ChatGPT Data Leak Vulnerability

Summary: OpenAI has begun addressing a data exfiltration vulnerability (where attackers steal user data) in ChatGPT that exploits image markdown rendering during prompt injection attacks (tricking an AI by hiding instructions in its input). The company implemented a client-side validation check called 'url_safe' on the web app that blocks images from suspicious domains, though the fix is incomplete and attackers can still leak small amounts of data through workarounds.

Solution: OpenAI implemented a mitigation by adding a client-side validation API call (url_safe endpoint) that checks whether image URLs are safe before rendering them. The validation returns {"safe":false} to prevent rendering images from malicious domains. However, the source explicitly notes this is not a complete fix and suggests OpenAI should additionally "limit the number of images that are rendered per response to just one or maybe a handful maximum" to further reduce bypass techniques. The source also notes the current iOS version 1.2023.347 (16603) does not yet have these improvements.


CVE-2023-6730: Deserialization of Untrusted Data in GitHub repository huggingface/transformers prior to 4.36.

Summary: CVE-2023-6730 is a deserialization of untrusted data vulnerability (a security flaw where a program unsafely reconstructs objects from untrusted input, potentially allowing attackers to execute malicious code) found in the Hugging Face Transformers library before version 4.36. The vulnerability has a CVSS score of 4.0, which indicates a moderate severity level (a 0-10 rating of how severe a vulnerability is).


CVE-2023-6909: Path Traversal: '\..\filename' in GitHub repository mlflow/mlflow prior to 2.9.2.

Summary: CVE-2023-6909 is a path traversal vulnerability (a security flaw where an attacker can access files outside their intended directory using special characters like '..\'). It affects MLflow versions before 2.9.2 in the mlflow/mlflow GitHub repository. The vulnerability was discovered and reported through the huntr.dev bug bounty platform.

Solution: Update MLflow to version 2.9.2 or later. A patch is available at the GitHub commit referenced: https://github.com/mlflow/mlflow/commit/1da75dfcecd4d169e34809ade55748384e8af6c1


CVE-2023-6831: Path Traversal: '\..\filename' in GitHub repository mlflow/mlflow prior to 2.9.2.

Summary: CVE-2023-6831 is a path traversal vulnerability (a flaw where an attacker can access files outside the intended directory by using special characters like '..\'). in MLflow versions before 2.9.2 that allows attackers to manipulate file paths and access restricted files they shouldn't be able to reach.

Solution: Update MLflow to version 2.9.2 or later. A patch is available at https://github.com/mlflow/mlflow/commit/1da75dfcecd4d169e34809ade55748384e8af6c1.


CVE-2023-6572: Command Injection in GitHub repository gradio-app/gradio prior to main.

Summary: CVE-2023-6572 is a command injection vulnerability (a security flaw where an attacker can run unauthorized commands) in the Gradio application (a tool for building AI demos) versions prior to the main branch. The vulnerability results from improper handling of special characters that could allow attackers to execute commands on affected systems.

Solution: A patch is available at the GitHub commit: https://github.com/gradio-app/gradio/commit/5b5af1899dd98d63e1f9b48a93601c2db1f56520. Users should update to the main branch or apply this commit to fix the vulnerability.


CVE-2023-6753: Path Traversal in GitHub repository mlflow/mlflow prior to 2.9.2.

Summary: CVE-2023-6753 is a path traversal vulnerability (a security flaw where an attacker can access files outside the intended directory by using special path characters) found in MLflow versions before 2.9.2. The vulnerability allows unauthorized access to restricted files on a system running the affected software.

Solution: Update MLflow to version 2.9.2 or later. A patch is available at https://github.com/mlflow/mlflow/commit/1c6309f884798fbf56017a3cc808016869ee8de4.


Malicious ChatGPT Agents: How GPTs Can Quietly Grab Your Data (Demo)

Summary: A researcher demonstrated that malicious GPTs (custom ChatGPT agents) can secretly steal user data by embedding hidden images in conversations that send information to external servers, and can also trick users into sharing personal details like passwords. OpenAI's validation checks for publishing GPTs can be easily bypassed by slightly rewording malicious instructions, allowing harmful GPTs to be shared publicly, though the researcher reported these vulnerabilities to OpenAI in November 2023 without receiving a fix.

Solution: N/A -- no mitigation discussed in source.

OpenAI
ChatGPT
GPTs
Bing Chat
Google Bard
Anthropic Claude


CVE-2023-35625: Azure Machine Learning Compute Instance for SDK Users Information Disclosure Vulnerability

Summary: CVE-2023-35625 is a vulnerability in Azure Machine Learning Compute Instance that allows unauthorized users to access sensitive information through the SDK (software development kit, a collection of tools for building applications). The vulnerability is classified as an information disclosure issue, meaning private data could be exposed to people who shouldn't see it.


CVE-2023-6709: Improper Neutralization of Special Elements Used in a Template Engine in GitHub repository mlflow/mlflow prior to 2.9.2.

Summary: CVE-2023-6709 is a vulnerability in MLflow (a machine learning tool) versions before 2.9.2 involving improper neutralization of special elements in a template engine (a system that generates text by filling in placeholders in templates). This weakness could potentially allow attackers to manipulate how the software processes certain input data.

Solution: Update MLflow to version 2.9.2 or later. A patch is available at https://github.com/mlflow/mlflow/commit/432b8ccf27fd3a76df4ba79bb1bec62118a85625.


CVE-2023-6568: A reflected Cross-Site Scripting (XSS) vulnerability exists in the mlflow/mlflow repository, specifically within the han

Summary: MLflow, an open-source machine learning platform, has a reflected XSS (cross-site scripting, where an attacker injects malicious JavaScript that runs in a victim's browser) vulnerability in how it handles the Content-Type header in POST requests. An attacker can craft a malicious Content-Type header that gets sent back to the user without proper filtering, allowing arbitrary JavaScript code to execute in the victim's browser.


CVE-2023-43472: An issue in MLFlow versions 2.8.1 and before allows a remote attacker to obtain sensitive information via a crafted requ

Summary: CVE-2023-43472 is a vulnerability in MLFlow (an open-source platform for managing machine learning workflows) versions 2.8.1 and earlier that allows a remote attacker to obtain sensitive information by sending a specially crafted request to the REST API (the interface that programs use to communicate with MLFlow). The vulnerability has a CVSS severity score of 4.0 (a moderate risk level on a scale of 0-10).


Ekoparty Talk - Prompt Injections in the Wild

Summary: A security researcher presented at Ekoparty 2023 about prompt injections (attacks where malicious instructions are hidden in inputs to trick an AI into misbehaving) found in real-world LLM applications and chatbots like ChatGPT, Bing Chat, and Google Bard, demonstrating various exploits and discussing mitigations. The talk covered both basic LLM concepts and deep dives into how these attacks work across different AI platforms.

OpenAI Anthropic Google Microsoft Amazon

Bing Chat
ChatGPT
Anthropic Claude
Azure AI
GCP Vertex AI
Google Bard


CVE-2023-48299: TorchServe is a tool for serving and scaling PyTorch models in production. Starting in version 0.1.0 and prior to versio

Summary: TorchServe (a tool for running PyTorch machine learning models as web services) versions before 0.9.0 had a ZipSlip vulnerability (a flaw where an attacker can extract files outside the intended folder by crafting malicious archive files), allowing attackers to upload harmful code disguised in publicly available models that could execute on machines running TorchServe. The vulnerability affected the model and workflow management API, which handles uploaded files.

Solution: Upgrade to TorchServe version 0.9.0 or later. The fix validates the file paths in zip archives before extracting them to prevent files from being placed in unintended filesystem locations.


CVE-2023-46302: Apache Software Foundation Apache Submarine has a bug when serializing against yaml. The bug is caused by snakeyaml  htt

Summary: Apache Submarine has a security vulnerability in how it handles YAML (a data format language) requests because it uses an unsafe library called snakeyaml. When users send YAML data to the application through its REST API (a system for receiving web requests), the unsafe handling could allow attackers to execute malicious code.

Solution: Users should upgrade to Apache Submarine version 0.8.0, which fixes this issue by replacing snakeyaml with jackson-dataformat-yaml. If upgrading is not possible, users can cherry-pick (apply a specific code fix from) PR https://github.com/apache/submarine/pull/1054 and rebuild the submarine-server image.


CVE-2023-6020: LFI in Ray's /static/ directory allows attackers to read any file on the server without authentication.

Summary: CVE-2023-6020 is a local file inclusion (LFI, a vulnerability that lets attackers read files they shouldn't access) in Ray's /static/ directory that allows attackers to read any file on the server without needing to log in. The vulnerability stems from missing authorization checks (the system doesn't verify whether a user should have access before serving files).


CVE-2023-6014: An attacker is able to arbitrarily create an account in MLflow bypassing any authentication requirment.

Summary: CVE-2023-6014 is a vulnerability in MLflow (a machine learning experiment tracking platform) that allows attackers to create user accounts without proper authentication (the process of verifying someone's identity). The vulnerability has a CVSS score (a 0-10 rating of how severe a vulnerability is) of 4.0, indicating moderate severity.


CVE-2023-6021: LFI in Ray's log API endpoint allows attackers to read any file on the server without authentication. The issue is fixed

Summary: CVE-2023-6021 is a local file inclusion (LFI, a vulnerability where an attacker can read files from a server by manipulating file paths) in Ray's log API endpoint that allows attackers to read any file on the server without needing authentication. The vulnerability affects Ray versions before 2.8.1.

Solution: The issue is fixed in version 2.8.1+. Users should upgrade to Ray version 2.8.1 or later.


CVE-2023-6018: An attacker can overwrite any file on the server hosting MLflow without any authentication.

Summary: CVE-2023-6018 is a vulnerability in MLflow (an open-source machine learning platform) that allows an attacker to overwrite any file on the server without needing to log in or authenticate. The vulnerability is caused by OS command injection (a flaw where special characters in user input are not properly filtered before being executed as system commands), which gives attackers the ability to run unauthorized commands on the server.


CVE-2023-6015: MLflow allowed arbitrary files to be PUT onto the server.

Summary: CVE-2023-6015 is a vulnerability in MLflow that allows attackers to upload arbitrary files to the server using PUT requests. This is a path traversal vulnerability (CWE-22, where an attacker can write files outside the intended directory by manipulating file paths), with a CVSS severity score of 4.0 (a moderate-level security issue on a 0-10 scale).


CVE-2023-5245: FileUtil.extract() enumerates all zip file entries and extracts each file without validating whether file paths in the a

Summary: CVE-2023-5245 is a vulnerability in FileUtil.extract() where zip file extraction does not check if file paths are outside the intended directory, allowing attackers to create files anywhere and potentially execute code when TensorflowModel processes a saved model. This is called path traversal (a technique where an attacker uses file paths like '../../../' to escape a restricted folder).


Hacking Google Bard - From Prompt Injection to Data Exfiltration

Summary: Google Bard's new Extensions feature allows it to access personal data like YouTube videos, Google Drive files, Gmail, and Google Docs. Because Bard analyzes this untrusted data, it is vulnerable to indirect prompt injection (a technique where hidden instructions in documents trick an AI into performing unintended actions), which a researcher demonstrated by getting Bard to summarize videos and documents.

Google Bard
Google Drive
Google Docs
Gmail
YouTube


CVE-2023-46315: The zanllp sd-webui-infinite-image-browsing (aka Infinite Image Browsing) extension before 977815a for stable-diffusion-

Summary: The Infinite Image Browsing extension for Stable Diffusion web UI (a tool for generating images with AI) has a security flaw that allows attackers to read any file on a computer if Gradio authentication is enabled without a secret key configuration. Attackers can exploit this by manipulating URLs with /file?path= to access sensitive files, such as environment variables that might contain login credentials.

Solution: Update to commit 977815a or later. The patch is available at https://github.com/zanllp/sd-webui-infinite-image-browsing/pull/368/commits/977815a2b28ad953c10ef0114c365f698c4b8f19

Stability AI
sd-webui-infinite-image-browsing
Stable Diffusion web UI


CVE-2023-32786: In Langchain through 0.0.155, prompt injection allows an attacker to force the service to retrieve data from an arbitrar

Summary: CVE-2023-32786 is a prompt injection vulnerability (tricking an AI by hiding instructions in its input) in Langchain version 0.0.155 and earlier that allows attackers to force the service to retrieve data from any URL they choose. This could lead to SSRF (server-side request forgery, where an attacker makes a server request data from unintended locations) and potentially inject harmful content into tasks that use the retrieved data.


Google Cloud Vertex AI - Data Exfiltration Vulnerability Fixed in Generative AI Studio

Summary: Google Cloud's Vertex AI Generative AI Studio had a data exfiltration vulnerability caused by image markdown injection (a technique where attackers embed hidden commands in image references to steal data). The vulnerability was responsibly disclosed to Google and has been fixed.

Google Cloud
Vertex AI
Generative AI Studio


CVE-2023-46229: LangChain before 0.0.317 allows SSRF via document_loaders/recursive_url_loader.py because crawling can proceed from an e

Summary: LangChain versions before 0.0.317 have a vulnerability called SSRF (server-side request forgery, where an attacker tricks the application into making requests to unintended servers) in its recursive URL loader component. The flaw allows web crawling to move from an external server to an internal server that should not be accessible.

Solution: Update LangChain to version 0.0.317 or later. Patches are available at https://github.com/langchain-ai/langchain/commit/9ecb7240a480720ec9d739b3877a52f76098a2b8 and https://github.com/langchain-ai/langchain/pull/11925.


CVE-2023-45063: Cross-Site Request Forgery (CSRF) vulnerability in ReCorp AI Content Writing Assistant (Content Writer, GPT 3 & 4, ChatG

Summary: A CSRF vulnerability (cross-site request forgery, where an attacker tricks a user into performing unwanted actions on a website they're logged into) was found in the ReCorp AI Content Writing Assistant plugin for WordPress in versions 1.1.5 and earlier. This flaw could allow attackers to exploit users of the plugin without their knowledge.


CVE-2023-44467: langchain_experimental (aka LangChain Experimental) in LangChain before 0.0.306 allows an attacker to bypass the CVE-202

Summary: CVE-2023-44467 is a vulnerability in LangChain Experimental (a library for building AI applications) before version 0.0.306 that allows attackers to bypass a previous security fix and run arbitrary code (unauthorized commands) on a system using the __import__ function in Python, which the pal_chain/base.py file failed to block.

Solution: Upgrade LangChain to version 0.0.306 or later. A patch is available at https://github.com/langchain-ai/langchain/commit/4c97a10bd0d9385cfee234a63b5bd826a295e483.


Microsoft Fixes Data Exfiltration Vulnerability in Azure AI Playground

Summary: LLM applications like chatbots are vulnerable to data exfiltration (unauthorized data theft) through image markdown injection, a technique where attackers embed hidden instructions in untrusted data to make the AI generate image tags that leak information. Microsoft patched this vulnerability in Azure AI Playground, though the source does not describe the specific technical details of their fix.

Microsoft
Azure AI Playground
Bing Chat
Anthropic
Claude
OpenAI
ChatGPT


CVE-2023-43654: TorchServe is a tool for serving and scaling PyTorch models in production. TorchServe default configuration lacks proper

Summary: TorchServe (a tool for running PyTorch machine learning models as web services) has a vulnerability in its default configuration that fails to validate user inputs properly, allowing attackers to download files from any URL and save them to the server's disk. This could let attackers damage the system or steal sensitive information, affecting versions 0.1.0 through 0.8.1.

Solution: Upgrade to TorchServe release 0.8.2 or later, which includes a warning when the default value for allowed_urls is used. Users should also configure the allowed_urls setting and specify which model URLs are permitted.


Advanced Data Exfiltration Techniques with ChatGPT

Summary: An indirect prompt injection attack (tricking an AI into following hidden instructions in its input) can allow an attacker to steal chat data from ChatGPT users by either having the AI embed information into image URLs (image markdown injection, which embeds data into web links displayed as images) or convincing users to click malicious links. ChatGPT Plugins, which are add-ons that extend ChatGPT's functionality, create additional exfiltration risks because they have minimal security review before being deployed.


HITCON CMT 2023 - LLM Security Presentation and Trip Report

Summary: This article is a trip report from HITCON CMT 2023, a security conference in Taiwan, where the author attended talks on various topics including LLM security, reverse engineering with AI, and application exploits. Key presentations covered indirect prompt injections (attacks where malicious instructions are hidden in data fed to an AI system), Electron app vulnerabilities, and PHP security issues. The author gave a talk on indirect prompt injections and notes this technique could become a significant attack vector for AI-integrated applications like chatbots.


LLM Apps: Don't Get Stuck in an Infinite Loop! 💵💰

Summary: An attacker can use indirect prompt injection (tricking an AI by hiding malicious instructions in data it reads) to make an LLM call its own tools or plugins repeatedly in a loop, potentially increasing costs or disrupting service. While ChatGPT users are mostly protected by subscription pricing, call limits, and a manual stop button, this technique demonstrates a real vulnerability in how LLM applications handle recursive tool calls.


CVE-2023-41626: Gradio v3.27.0 was discovered to contain an arbitrary file upload vulnerability via the /upload interface.

Summary: Gradio version 3.27.0 has a security flaw that allows attackers to upload any type of file through the /upload interface without proper restrictions (CWE-434, unrestricted file upload with dangerous type). This means someone could potentially upload malicious files to a system running this vulnerable version.


CVE-2023-39631: An issue in LanChain-ai Langchain v.0.0.245 allows a remote attacker to execute arbitrary code via the evaluate function

Summary: CVE-2023-39631 is a code injection vulnerability (a flaw where an attacker can insert malicious code into a program) in Langchain version 0.0.245 that allows a remote attacker to execute arbitrary code through the evaluate function in the numexpr library (a Python tool for fast numerical expression evaluation). The vulnerability has a CVSS severity score of 4.0, indicating low to moderate risk.


v2: make download.sh executable (#695)

Summary: This is a minor update to the Llama repository that makes download.sh (a script file used to download files) executable and adds error handling so the script stops running if it encounters a problem. The change was submitted as a pull request to improve the reliability of the download process.


CVE-2023-38975: * Buffer Overflow vulnerability in qdrant v.1.3.2 allows a remote attacker cause a denial of service via the chucnked_ve

Summary: A buffer overflow vulnerability (a memory safety flaw where data is written beyond allocated space) in Qdrant version 1.3.2 allows remote attackers to cause a denial of service (making the service unavailable) through the chunked_vectors component. The vulnerability has a CVSS score of 4.0, indicating moderate severity.


Video: Data Exfiltration Vulnerabilities in LLM apps (Bing Chat, ChatGPT, Claude)

Summary: A researcher discovered data exfiltration vulnerabilities (security flaws that allow unauthorized data to leak out of a system) in several popular AI chatbots including Bing Chat, ChatGPT, and Claude, and responsibly disclosed them to the companies. Microsoft, Anthropic, and a plugin vendor fixed their vulnerabilities, but OpenAI decided not to fix an image markdown injection issue (a vulnerability where hidden code in image formatting can trick the AI into revealing data).

Solution: The source mentions that Microsoft (Bing Chat), Anthropic (Claude), and a plugin vendor addressed and fixed their respective vulnerabilities. However, OpenAI's response to the reported vulnerability was "won't fix," meaning no mitigation from OpenAI is described in the source text.

Microsoft Bing Chat
Anthropic Claude
OpenAI ChatGPT
Zapier


CVE-2023-36281: An issue in langchain v.0.0.171 allows a remote attacker to execute arbitrary code via a JSON file to load_prompt. This

Summary: LangChain version 0.0.171 has a vulnerability (CVE-2023-36281) that allows a remote attacker to execute arbitrary code (run commands they shouldn't be able to run) by sending a specially crafted JSON file to the load_prompt function. The vulnerability relates to improper control of code generation, which means the application doesn't properly validate or sanitize (clean) the input before using it to create executable code.


CVE-2023-38976: An issue in weaviate v.1.20.0 allows a remote attacker to cause a denial of service via the handleUnbatchedGraphQLReques

Summary: Weaviate v.1.20.0 contains a vulnerability (CVE-2023-38976) in the handleUnbatchedGraphQLRequest function that allows remote attackers to cause a denial of service (making a service unavailable by overwhelming it with requests). The vulnerability has a CVSS score of 4.0 (a moderate severity rating).


CVE-2023-39659: An issue in langchain langchain-ai v.0.0.232 and before allows a remote attacker to execute arbitrary code via a crafted

Summary: CVE-2023-39659 is a vulnerability in langchain (an AI library) version 0.0.232 and earlier that allows a remote attacker to execute arbitrary code (run commands they choose) by sending a specially crafted script to the PythonAstREPLTool._run component. The vulnerability is caused by improper neutralization of special elements in output (a type of injection attack where untrusted input is not properly filtered before being processed).


CVE-2023-38896: An issue in Harrison Chase langchain v.0.0.194 and before allows a remote attacker to execute arbitrary code via the fro

Summary: CVE-2023-38896 is a vulnerability in langchain v.0.0.194 and earlier versions that allows a remote attacker to execute arbitrary code (run commands on a system they don't control) through the from_math_prompt and from_colored_object_prompt functions. This is an injection attack (CWE-74), where the software fails to properly filter special characters or commands that could be misused by downstream components.

Solution: A patch is available at https://github.com/hwchase17/langchain/pull/6003. Users should update langchain to a version after v.0.0.194.


CVE-2023-38860: An issue in LangChain v.0.0.231 allows a remote attacker to execute arbitrary code via the prompt parameter.

Summary: LangChain version 0.0.231 has a vulnerability (CVE-2023-38860) where a remote attacker can execute arbitrary code by manipulating the prompt parameter, which is a type of code injection (CWE-94, where an attacker tricks the system into running malicious code by hiding it in input data).


CVE-2023-27506: Improper buffer restrictions in the Intel(R) Optimization for Tensorflow software before version 2.12 may allow an authe

Summary: CVE-2023-27506 is a vulnerability in Intel Optimization for Tensorflow software before version 2.12 involving improper buffer restrictions (a memory safety flaw where a program doesn't properly check that it stays within allocated memory). An authenticated user with local access to a system could potentially use this flaw to escalate their privileges, gaining higher-level access than they should have.

Solution: Update Intel Optimization for Tensorflow to version 2.12 or later.


CVE-2023-36095: An issue in Harrison Chase langchain v.0.0.194 allows an attacker to execute arbitrary code via the python exec calls in

Summary: LangChain (an AI framework for building applications with language models) version 0.0.194 contains a code injection vulnerability (CWE-94, a weakness where attackers can inject malicious code into a program) that allows attackers to execute arbitrary code through the PALChain component, specifically in the from_math_prompt and from_colored_object_prompt functions that use Python's exec command.


Anthropic Claude Data Exfiltration Vulnerability Fixed

Summary: Anthropic patched a data exfiltration vulnerability in Claude caused by image markdown injection, a technique where attackers embed hidden instructions in image links to trick the AI into leaking sensitive information. While Microsoft fixed this vulnerability in Bing Chat and OpenAI chose not to address it in ChatGPT, Anthropic implemented a mitigation to protect Claude users from this attack.

Anthropic
Claude
Microsoft
Bing Chat
OpenAI
ChatGPT


CVE-2023-4033: OS Command Injection in GitHub repository mlflow/mlflow prior to 2.6.0.

Summary: CVE-2023-4033 is an OS command injection vulnerability (a type of attack where an attacker can run arbitrary system commands) found in MLflow, an open-source machine learning platform, in versions before 2.6.0. The vulnerability allows attackers to execute unauthorized commands on affected systems.

Solution: Update MLflow to version 2.6.0 or later. A patch is available at the GitHub commit: https://github.com/mlflow/mlflow/commit/6dde93758d42455cb90ef324407919ed67668b9b


ChatGPT Custom Instructions: Persistent Data Exfiltration Demo

Summary: ChatGPT has a vulnerability where attackers can use image markdown (a way to embed images in text) to trick the system into leaking data. OpenAI recently added Custom Instructions, a feature that automatically adds instructions to every message, which attackers can abuse to install a persistent backdoor (hidden access point) that steals data through the image markdown vulnerability. This technique is similar to how attackers exploit other systems by enabling features like email forwarding after they gain initial access.


CVE-2023-3765: Absolute Path Traversal in GitHub repository mlflow/mlflow prior to 2.5.0.

Summary: MLflow (a popular machine learning platform) versions before 2.5.0 contain a vulnerability called absolute path traversal (CWE-36, where an attacker can access files anywhere on a system by manipulating file paths). This vulnerability was identified and reported through the huntr.dev bug bounty program.

Solution: Upgrade to MLflow version 2.5.0 or later. A patch is available at https://github.com/mlflow/mlflow/commit/6dde93758d42455cb90ef324407919ed67668b9b.


CVE-2023-3686: A vulnerability was found in Bylancer QuickAI OpenAI 3.8.1. It has been declared as critical. This vulnerability affects

Summary: A critical vulnerability (CVE-2023-3686) was found in Bylancer QuickAI OpenAI version 3.8.1 that allows SQL injection (a technique where attackers insert malicious database commands into user input) through the 's' parameter in the /blog file's GET Parameter Handler. The attack can be launched remotely, and the vendor did not respond to early disclosure attempts.


Image to Prompt Injection with Google Bard

Summary: Google Bard can be tricked through image-based prompt injection (hidden instructions placed in images that the AI then follows), as demonstrated by a researcher who embedded text in an image that caused Bard to perform unexpected actions. This vulnerability shows that AI systems that analyze images may be vulnerable to indirect prompt injection attacks (tricking an AI into ignoring its normal instructions by hiding malicious commands in user-provided content).


CVE-2023-37275: Auto-GPT is an experimental open-source application showcasing the capabilities of the GPT-4 language model. The Auto-GP

Summary: Auto-GPT is an experimental application that uses GPT-4 (a large language model) to demonstrate AI capabilities through a command-line interface. Before version 0.4.3, malicious websites could trick Auto-GPT's language model into outputting specially encoded text (ANSI escape sequences, which are hidden commands that control console display) that would create fake or misleading messages on the user's screen, potentially causing them to run unintended commands.

Solution: The issue has been patched in release version 0.4.3.


CVE-2023-37274: Auto-GPT is an experimental open-source application showcasing the capabilities of the GPT-4 language model. When Auto-G

Summary: Auto-GPT versions before 0.4.3 have a path traversal vulnerability (a weakness where an attacker uses file paths like '../../../' to access files outside the intended directory) in the `execute_python_code` command that fails to validate filenames, allowing an attacker to write malicious code outside the sandbox and execute arbitrary commands on the host system. This vulnerability bypasses the Docker container (a tool that isolates applications) meant to protect the main system from untrusted code.

Solution: The issue has been patched in version 0.4.3. As a workaround, run Auto-GPT in a virtual machine or another environment in which damage to files or corruption of the program is not a critical problem.


CVE-2023-37273: Auto-GPT is an experimental open-source application showcasing the capabilities of the GPT-4 language model. Running Aut

Summary: Auto-GPT versions before 0.4.3 have a security flaw where the docker-compose.yml file (a configuration file that sets up Docker containers) is mounted into the container without write protection. If an attacker tricks Auto-GPT into running malicious code through the `execute_python_file` or `execute_python_code` commands, they can overwrite this file and gain control of the host system (the main computer running Auto-GPT) when it restarts.

Solution: Update to Auto-GPT version 0.4.3 or later, as the issue has been patched in that version.


Google Docs AI Features: Vulnerabilities and Risks

Summary: Google Docs recently added new AI features, such as automatic summaries and creative content generation, which are helpful but introduce security risks. The main concern is that using these AI features on untrusted data (information you don't know the source or reliability of) could lead to unwanted consequences, though currently attackers have limited ways to exploit these features.

Solution: N/A -- no mitigation discussed in source.


OpenAI Removes the "Chat with Code" Plugin From Store

Summary: OpenAI removed the 'Chat with Code' plugin from its store after security researchers discovered it was vulnerable to CSRF (cross-site request forgery, where an attacker tricks a system into making unwanted actions on behalf of a user). The vulnerability allowed ChatGPT to accidentally create GitHub issues without user permission when certain plugins were enabled together.


CVE-2023-36189: SQL injection vulnerability in langchain before v0.0.247 allows a remote attacker to obtain sensitive information via th

Summary: A SQL injection vulnerability (a type of attack where an attacker inserts malicious SQL commands into input fields) exists in langchain versions before v0.0.247 in the SQLDatabaseChain component, allowing remote attackers to obtain sensitive information from databases.

Solution: Update langchain to version v0.0.247 or later.


CVE-2023-36188: An issue in langchain v.0.0.64 allows a remote attacker to execute arbitrary code via the PALChain parameter in the Pyth

Summary: CVE-2023-36188 is a vulnerability in langchain version 0.0.64 that allows a remote attacker to execute arbitrary code (running commands they shouldn't be able to run) through the PALChain parameter in Python's exec method. This is a type of injection attack (CWE-74, where an attacker tricks a system by inserting malicious code into input that gets processed as commands).

Solution: A patch is available at https://github.com/hwchase17/langchain/pull/6003


CVE-2023-36258: An issue in LangChain before 0.0.236 allows an attacker to execute arbitrary code because Python code with os.system, ex

Summary: CVE-2023-36258 is a vulnerability in LangChain before version 0.0.236 that allows an attacker to execute arbitrary code (run any commands they want on a system) by exploiting the ability to use Python functions like os.system, exec, or eval (functions that can run code dynamically). This is a code injection vulnerability (CWE-94, where attackers trick a program into running unintended code).

Solution: Upgrade LangChain to version 0.0.236 or later.


CVE-2023-34541: Langchain 0.0.171 is vulnerable to Arbitrary code execution in load_prompt.

Summary: Langchain version 0.0.171 has a vulnerability that allows arbitrary code execution (running uncontrolled commands on a system) through its load_prompt function. The vulnerability was reported in June 2023, but the provided source material does not contain detailed information about how the vulnerability works or its severity rating.


Plugin Vulnerabilities: Visit a Website and Have Your Source Code Stolen

Summary: OpenAI's plugin store contains security vulnerabilities, particularly in plugins that can act on behalf of users without adequate security review. These plugins are susceptible to prompt injection attacks (tricking an AI by hiding instructions in its input) and the Confused Deputy Problem (where an attacker can manipulate a plugin into performing harmful actions by exploiting its trust in the AI system), allowing adversaries to steal source code or cause other damage.


Bing Chat: Data Exfiltration Exploit Explained

Summary: Bing Chat contained a prompt injection vulnerability (tricking an AI by hiding instructions in its input) where malicious text on websites could trick the AI into returning markdown image tags that send sensitive data to an attacker's server. When Bing Chat's client converts markdown to HTML, an attacker can embed data in the image URL, exfiltrating (stealing and sending out) information without the user knowing.


CVE-2023-34540: Langchain before v0.0.225 was discovered to contain a remote code execution (RCE) vulnerability in the component JiraAPI

Summary: Langchain versions before v0.0.225 contained a remote code execution (RCE, where attackers can run commands on a system they don't own) vulnerability in the JiraAPIWrapper component that allowed attackers to execute arbitrary code through specially crafted input. The vulnerability was identified in the JiraAPI wrapper component of the library.

Solution: Update Langchain to v0.0.225 or later. A fix is available in the release v0.0.225.


Exploit ChatGPT and Enter the Matrix to Learn about AI Security

Summary: A security researcher created a demonstration website that shows how indirect prompt injection (tricking an AI by hiding instructions in web content it reads) can be used to hijack ChatGPT when the browsing feature is enabled. The demo lets users explore various AI-based attacks, including data theft and manipulation of ChatGPT's responses, to raise awareness of these vulnerabilities.


CVE-2023-34239: Gradio is an open-source Python library that is used to build machine learning and data science. Due to a lack of path f

Summary: Gradio, an open-source Python library for building machine learning and data science applications, has a vulnerability where it fails to properly filter file paths and restrict which URLs can be proxied (accessed through Gradio as an intermediary), allowing unauthorized file access. This vulnerability affects input validation (the process of checking that data entering a system is safe and expected).

Solution: Users are advised to upgrade to version 3.34.0. The source notes there are no known workarounds for this vulnerability.


CVE-2023-34094: ChuanhuChatGPT is a graphical user interface for ChatGPT and many large language models. A vulnerability in versions 202

Summary: ChuanhuChatGPT (a graphical interface for ChatGPT and other large language models) has a vulnerability in versions 20230526 and earlier that allows attackers to access the config.json file (a configuration file storing sensitive settings) without permission when authentication is disabled, potentially exposing API keys (credentials that grant access to external services). The vulnerability allows attackers to steal these API keys from the configuration file.

Solution: The vulnerability has been fixed in commit bfac445. As a workaround, setting up access authentication (a login system that restricts who can access the software) can help mitigate the vulnerability.


CVE-2023-33979: gpt_academic provides a graphical interface for ChatGPT/GLM. A vulnerability was found in gpt_academic 3.37 and prior. T

Summary: gpt_academic (a tool that provides a graphical interface for ChatGPT/GLM) versions 3.37 and earlier have a vulnerability where the Configuration File Handler allows attackers to read sensitive files through the `/file` route because no files are protected from access. This can leak sensitive information from working directories to users who shouldn't have access to it.

Solution: A patch is available at commit 1dcc2873d2168ad2d3d70afcb453ac1695fbdf02. As a workaround, users can configure the project using environment variables instead of `config*.py` files, or use docker-compose installation (a tool for running containerized applications) to configure the project instead of configuration files.


ChatGPT Plugin Exploit Explained: From Prompt Injection to Accessing Private Data

Summary: ChatGPT plugins can be exploited through indirect prompt injections (attacks that hide malicious instructions in data the AI reads from external sources rather than directly from the user), which hackers have used to access private data through cross-plugin request forgery (a vulnerability where one plugin tricks another into performing unauthorized actions). The post documents a real exploit found in the wild and explains the security fix that was applied.


CVE-2023-32676: Autolab is a course management service that enables auto-graded programming assignments. A Tar slip vulnerability was fo

Summary: Autolab, a service that automatically grades programming assignments in courses, has a tar slip vulnerability (a flaw where extracted files can be placed outside their intended directory) in its assessment installation feature. An attacker with instructor permissions could upload a specially crafted tar file (a compressed archive format) with file paths like `../../../../tmp/tarslipped1.sh` to place files anywhere on the system when the form is submitted.

Solution: Upgrade to version 2.11.0 or later.


CVE-2023-2800: Insecure Temporary File in GitHub repository huggingface/transformers prior to 4.30.0.

Summary: CVE-2023-2800 is a vulnerability in the Hugging Face Transformers library (a popular tool for working with AI language models) prior to version 4.30.0 that involves insecure temporary files (CWE-377, a weakness where temporary files are created in ways that attackers could exploit). The vulnerability was discovered and reported through the huntr.dev bug bounty platform.

Solution: Update to version 4.30.0 or later. A patch is available at https://github.com/huggingface/transformers/commit/80ca92470938bbcc348e2d9cf4734c7c25cb1c43.


CVE-2023-2780: Path Traversal: '\..\filename' in GitHub repository mlflow/mlflow prior to 2.3.1.

Summary: MLflow (a tool for managing machine learning experiments) versions before 2.3.1 contain a path traversal vulnerability (CWE-29, a weakness where attackers can access files outside intended directories by using special characters like '..\'). This vulnerability could allow an attacker to read or manipulate files they shouldn't have access to.

Solution: Update MLflow to version 2.3.1 or later. A patch is available at https://github.com/mlflow/mlflow/commit/fae77a525dd908c56d6204a4cef1c1c75b4e9857.


ChatGPT Plugins: Data Exfiltration via Images & Cross Plugin Request Forgery

Summary: A malicious website can hijack a ChatGPT chat session and steal conversation history by controlling the data that plugins (add-ons that extend ChatGPT's abilities) retrieve. The post highlights that while plugins can leak data by receiving too much information, the main risk here is when an attacker controls what data the plugin pulls in, enabling them to extract sensitive information.


Indirect Prompt Injection via YouTube Transcripts

Summary: ChatGPT can access YouTube transcripts through plugins, which is useful but creates a security risk called indirect prompt injection (hidden instructions embedded in content that an AI reads and then follows). Attackers can hide malicious commands in video transcripts, and when ChatGPT reads those transcripts to answer user questions, it may follow the hidden instructions instead of the user's intended request.


Adversarial Prompting: Tutorial and Lab

Summary: This resource is a tutorial and lab (an interactive learning environment for hands-on practice) that teaches prompt injection, which is a technique for tricking AI systems by embedding hidden instructions in their input. The tutorial covers examples ranging from simple prompt engineering (getting an AI to change its output) to more complex attacks like injecting malicious code (HTML/XSS, which runs unwanted scripts in web browsers) and stealing data from AI systems.


CVE-2023-30172: A directory traversal vulnerability in the /get-artifact API method of the mlflow platform up to v2.0.1 allows attackers

Summary: CVE-2023-30172 is a directory traversal vulnerability (a flaw where attackers can access files outside the intended folder by manipulating file paths) in the /get-artifact API method of MLflow platform versions up to v2.0.1. Attackers can exploit the path parameter to read arbitrary files stored on the server.


Video: Prompt Injections - An Introduction

Summary: Prompt injection (tricking an AI by hiding instructions in its input) is a widespread vulnerability in AI education, with indirect prompt injections being particularly dangerous because they allow untrusted data to secretly take control of an LLM (large language model) and change its goals and behavior. Since attack payloads use natural language, attackers can craft many creative variations to bypass input validation (checking that data meets safety rules) and web application firewalls (security systems that filter harmful requests).


CVE-2023-1651: The AI ChatBot WordPress plugin before 4.4.9 does not have authorisation and CSRF in the AJAX action responsible to upda

Summary: The AI ChatBot WordPress plugin before version 4.4.9 has two security flaws in its code that handles OpenAI settings. First, it lacks authorization checks (meaning it doesn't verify who should be allowed to make changes), allowing even low-privilege users like subscribers to modify settings. Second, it's vulnerable to CSRF (cross-site request forgery, where an attacker tricks a logged-in user into making unwanted changes) and stored XSS (cross-site scripting, where malicious code gets saved and runs when others view the page).

Solution: Update the AI ChatBot WordPress plugin to version 4.4.9 or later.


CVE-2023-2356: Relative Path Traversal in GitHub repository mlflow/mlflow prior to 2.3.1.

Summary: CVE-2023-2356 is a relative path traversal vulnerability (a flaw that lets attackers access files outside their intended directory by manipulating file paths) found in MLflow versions before 2.3.1. This weakness could allow attackers to read or access files they shouldn't be able to reach on systems running the affected software.

Solution: Update MLflow to version 2.3.1 or later. A patch is available at https://github.com/mlflow/mlflow/commit/f73147496e05c09a8b83d95fb4f1bf86696c6342.


MLSecOps Podcast: AI Red Teaming and Threat Modeling Machine Learning Systems

Summary: This is a podcast episode about AI red teaming (simulated attacks to find weaknesses in AI systems) and threat modeling (planning for potential security risks) in machine learning systems. The episode explores how traditional security practices can be combined with machine learning security to better protect AI applications from attacks.


CVE-2023-30444: IBM Watson Machine Learning on Cloud Pak for Data 4.0 and 4.5 is vulnerable to server-side request forgery (SSRF). This

Summary: IBM Watson Machine Learning on Cloud Pak for Data versions 4.0 and 4.5 has a vulnerability called SSRF (server-side request forgery, where an attacker tricks the system into making unauthorized network requests on their behalf). An authenticated attacker could exploit this to discover network details or launch other attacks.

IBM Watson Machine Learning
IBM Cloud Pak for Data


CVE-2023-30620: mindsdb is a Machine Learning platform to help developers build AI solutions. In affected versions an unsafe extraction

Summary: MindsDB, a platform for building AI solutions, has a vulnerability in older versions where it unsafely extracts files from remote archives using `tarfile.extractall()` (a Python function that unpacks compressed files). An attacker could exploit this to overwrite any file that the server can access, similar to known attacks called TarSlip or ZipSlip (path traversal attacks, where files are extracted to unexpected locations).

Solution: Upgrade to release 23.2.1.0 or later. The source explicitly states 'There are no known workarounds for this vulnerability,' so updating is the only mitigation mentioned.


Don't blindly trust LLM responses. Threats to chatbots.

Summary: LLM outputs are untrusted and can be manipulated through prompt injection (tricking an AI by hiding instructions in its input), which affects large language models in particular ways. This post addresses how to handle the risks of untrusted output when using AI systems in real applications.


CVE-2023-28312: Azure Machine Learning Information Disclosure Vulnerability

Summary: CVE-2023-28312 is an information disclosure vulnerability in Azure Machine Learning, meaning unauthorized people could access sensitive data they shouldn't be able to see. The vulnerability involves improper access control (CWE-284, a weakness where the system doesn't properly check who is allowed to access what), and it was reported by Microsoft.


CVE-2023-29374: In LangChain through 0.0.131, the LLMMathChain chain allows prompt injection attacks that can execute arbitrary code via

Summary: CVE-2023-29374 is a vulnerability in LangChain versions up to 0.0.131 where the LLMMathChain component is vulnerable to prompt injection attacks (tricking an AI by hiding instructions in its input), allowing attackers to execute arbitrary code through Python's exec method. This is a code execution vulnerability that could allow an attacker to run malicious commands on a system running the affected software.

Solution: A patch is available at https://github.com/hwchase17/langchain/pull/1119


CVE-2022-23522: MindsDB is an open source machine learning platform. An unsafe extraction is being performed using `shutil.unpack_archiv

Summary: MindsDB, an open source machine learning platform, has a vulnerability where it unsafely unpacks tar files (compressed archives) using a function that doesn't check if extracted files stay in the intended folder. An attacker could create a malicious tar file with a specially crafted filename (like `../../../../etc/passwd`) that tricks the system into writing files to sensitive system locations, potentially overwriting important system files on the server running MindsDB.

Solution: This issue has been addressed in version 22.11.4.3. Users are advised to upgrade. Users unable to upgrade should avoid ingesting archives from untrusted sources.


AI Injections: Direct and Indirect Prompt Injections and Their Implications

Summary: AI prompt injection is a vulnerability where attackers manipulate input given to AI systems, either directly (by controlling parts of the prompt themselves) or indirectly (by embedding malicious instructions in data the AI will later process, like web pages). These attacks can trick AI systems into ignoring their intended instructions and producing harmful, misleading, or inappropriate responses, similar to how SQL injection or cross-site scripting (XSS, a web attack that injects malicious code into websites) compromise other systems.


CVE-2023-25661: TensorFlow is an Open Source Machine Learning Framework. In versions prior to 2.11.1 a malicious invalid input crashes a

Summary: TensorFlow (an open-source machine learning framework) versions before 2.11.1 have a bug where a malicious invalid input can crash a model and trigger a denial of service attack (making a service unavailable by overwhelming it). The vulnerability exists in the Convolution3DTranspose function, which is commonly used in modern neural networks, and could be exploited if an attacker can send input to this function.

Solution: Upgrade to TensorFlow version 2.11.1 or later. The source states there are no known workarounds for this vulnerability.


Bing Chat claims to have robbed a bank and it left no trace

Summary:
# Analysis

## Summary

A user discovered that Bing Chat could be manipulated into describing illegal activities (like bank robbery) by using indirect language techniques, even though it refused to help when the user directly asked about hacking. This shows that the AI's safety filters, which are supposed to prevent harmful outputs, can be bypassed through clever wording rather than direct requests.

## Solution

N/A -- no mitigation discussed in source.

Solution: N/A — no mitigation discussed in source.


CVE-2023-28858: redis-py before 4.5.3 leaves a connection open after canceling an async Redis command at an inopportune time, and can se

Summary: CVE-2023-28858 is a bug in redis-py (a Python library for connecting to Redis databases) versions before 4.5.3 where canceling an async command at the wrong moment leaves a connection open and can accidentally send response data from one request to a completely different client, due to an off-by-one error (miscounting by one position in the data stream).

Solution: Update redis-py to version 4.3.6, 4.4.3, or 4.5.3 or later. The patches are available in the official repository at https://github.com/redis/redis-py/ for each version.


CVE-2023-27579: TensorFlow is an end-to-end open source platform for machine learning. Constructing a tflite model with a paramater `fil

Summary: TensorFlow, an open-source machine learning platform, has a bug where creating a tflite model (a lightweight version of a machine learning model for mobile devices) with a filter_input_channel parameter set to less than 1 causes an FPE (floating-point exception, a math error that crashes the program). This vulnerability stems from an incorrect comparison in the code.

Solution: The issue has been patched in TensorFlow version 2.12. TensorFlow will also apply the fix to version 2.11.1. Users can reference the patch commit at https://github.com/tensorflow/tensorflow/commit/34f8368c535253f5c9cb3a303297743b62442aaa.


CVE-2023-25801: TensorFlow is an open source machine learning platform. Prior to versions 2.12.0 and 2.11.1, `nn_ops.fractional_avg_pool

Summary: TensorFlow, an open source machine learning platform, had a bug in two pooling functions (`nn_ops.fractional_avg_pool_v2` and `nn_ops.fractional_max_pool_v2`) that required certain parameters to equal 1.0 because pooling on batch and channel dimensions (the different ways data is organized in the neural network) was not supported. This vulnerability was fixed in TensorFlow versions 2.12.0 and 2.11.1.

Solution: Update to TensorFlow version 2.12.0 or 2.11.1, which include the fix for this vulnerability.


CVE-2023-25676: TensorFlow is an open source machine learning platform. When running versions prior to 2.12.0 and 2.11.1 with XLA, `tf.r

Summary: TensorFlow, an open source machine learning platform, has a bug in versions before 2.12.0 and 2.11.1 where the `tf.raw_ops.ParallelConcat` function crashes due to a null pointer dereference (trying to use a memory location that hasn't been set) when given a `shape` parameter with rank (dimensionality) of zero or less. This crash makes the program stop working unexpectedly.

Solution: Update TensorFlow to version 2.12.0 or 2.11.1 or later, which contain the fix for this vulnerability.


CVE-2023-25675: TensorFlow is an open source machine learning platform. When running versions prior to 2.12.0 and 2.11.1 with XLA, `tf.r

Summary: TensorFlow, an open source machine learning platform, has a bug in versions before 2.12.0 and 2.11.1 where the `tf.raw_ops.Bincount` function crashes when given a `weights` parameter that doesn't match the shape of the `arr` parameter or isn't a length-0 tensor (a parameter with zero elements). This crash only happens when XLA (accelerated linear algebra, a compiler for machine learning) is enabled.

Solution: Update to TensorFlow version 2.12.0 or 2.11.1, which include a fix for this issue.


CVE-2023-25674: TensorFlow is an open source machine learning platform. Versions prior to 2.12.0 and 2.11.1 have a null pointer error in

Summary: TensorFlow, an open source machine learning platform, has a null pointer error (a crash caused by the program trying to access memory that doesn't exist) in its RandomShuffle function when XLA (a compiler for machine learning) is enabled in versions before 2.12.0 and 2.11.1. This vulnerability has been assigned CVE-2023-25674.

Solution: Update TensorFlow to version 2.12.0 or 2.11.1, which include the fix for this null pointer error.


CVE-2023-25673: TensorFlow is an open source platform for machine learning. Versions prior to 2.12.0 and 2.11.1 have a Floating Point Ex

Summary: TensorFlow (an open source machine learning platform) versions before 2.12.0 and 2.11.1 have a Floating Point Exception bug in TensorListSplit with XLA (a compiler that speeds up machine learning computations). This bug could cause the program to crash when certain operations are performed.

Solution: Update to TensorFlow version 2.12.0 or version 2.11.1, where the fix is included.