AI Sec Watch

CVE-2023-44467 is a vulnerability in LangChain Experimental (a library for building AI applications) before version 0.0.306 that allows attackers to bypass a previous security fix and run arbitrary code (unauthorized commands) on a system using the __import__ function in Python, which the pal_chain/base.py file failed to block.

LLM applications like chatbots are vulnerable to data exfiltration (unauthorized data theft) through image markdown injection, a technique where attackers embed hidden instructions in untrusted data to make the AI generate image tags that leak information. Microsoft patched this vulnerability in Azure AI Playground, though the source does not describe the specific technical details of their fix.

TorchServe (a tool for running PyTorch machine learning models as web services) has a vulnerability in its default configuration that fails to validate user inputs properly, allowing attackers to download files from any URL and save them to the server's disk. This could let attackers damage the system or steal sensitive information, affecting versions 0.1.0 through 0.8.1.

An indirect prompt injection attack (tricking an AI into following hidden instructions in its input) can allow an attacker to steal chat data from ChatGPT users by either having the AI embed information into image URLs (image markdown injection, which embeds data into web links displayed as images) or convincing users to click malicious links. ChatGPT Plugins, which are add-ons that extend ChatGPT's functionality, create additional exfiltration risks because they have minimal security review before being deployed.

This article is a trip report from HITCON CMT 2023, a security conference in Taiwan, where the author attended talks on various topics including LLM security, reverse engineering with AI, and application exploits. Key presentations covered indirect prompt injections (attacks where malicious instructions are hidden in data fed to an AI system), Electron app vulnerabilities, and PHP security issues. The author gave a talk on indirect prompt injections and notes this technique could become a significant attack vector for AI-integrated applications like chatbots.

An attacker can use indirect prompt injection (tricking an AI by hiding malicious instructions in data it reads) to make an LLM call its own tools or plugins repeatedly in a loop, potentially increasing costs or disrupting service. While ChatGPT users are mostly protected by subscription pricing, call limits, and a manual stop button, this technique demonstrates a real vulnerability in how LLM applications handle recursive tool calls.

Gradio version 3.27.0 has a security flaw that allows attackers to upload any type of file through the /upload interface without proper restrictions (CWE-434, unrestricted file upload with dangerous type). This means someone could potentially upload malicious files to a system running this vulnerable version.

CVE-2023-39631 is a code injection vulnerability (a flaw where an attacker can insert malicious code into a program) in Langchain version 0.0.245 that allows a remote attacker to execute arbitrary code through the evaluate function in the numexpr library (a Python tool for fast numerical expression evaluation). The vulnerability has a CVSS severity score of 4.0, indicating low to moderate risk.

This is a minor update to the Llama repository that makes download.sh (a script file used to download files) executable and adds error handling so the script stops running if it encounters a problem. The change was submitted as a pull request to improve the reliability of the download process.