AI Sec Watch

The huggingface/transformers library has a vulnerability where attackers can run arbitrary code on a victim's machine by tricking them into loading a malicious checkpoint file. The problem occurs in the `load_repo_checkpoint()` function, which uses `pickle.load()` (a Python function that reconstructs objects from serialized data) on data that might come from untrusted sources, allowing attackers to execute remote code execution (RCE, where an attacker runs commands on a system they don't own).

Qdrant (a vector database software) has a vulnerability in its snapshot upload endpoint that allows attackers to upload files to any location on the server's filesystem through path traversal (using special file path sequences to access directories they shouldn't). This could let attackers execute arbitrary code on the server and damage the system's integrity and availability.

Gradio (a framework for building AI interfaces) has a vulnerability in its UploadButton component where it doesn't properly validate (check) user input, allowing attackers to read any file on the server by manipulating file paths sent to the `/queue/join` endpoint. This could let attackers steal sensitive files like SSH keys (credentials used for secure server access) and potentially execute arbitrary code on the system.

A vulnerability was found in the `safe_eval` function of the `llama_index` package that allows prompt injection (tricking an AI by hiding instructions in its input) to execute arbitrary code (running code an attacker chooses). The flaw exists because the input validation is insufficient, meaning the package doesn't properly check what data is being passed in, allowing attackers to bypass safety restrictions that were meant to prevent this type of attack.

Ollama before version 0.1.29 has a DNS rebinding vulnerability (a technique where an attacker tricks a system into connecting to a malicious server by manipulating how domain names are translated into addresses), which allows unauthorized remote access to its full API. This vulnerability could let an attacker interact with the language model, remove models, or cause a denial of service (making a system unavailable by overloading it with requests).

GPT Academic is a tool that provides interactive interfaces for large language models. Versions 3.64 through 3.73 have a vulnerability where the server deserializes untrusted data (processes data from users without verifying it's safe), which could allow attackers to execute code remotely on any exposed server. Any device running these vulnerable versions and accessible over the internet is at risk.

Google AI Studio had a vulnerability that allowed attackers to steal data through prompt injection (tricking an AI by hiding malicious instructions in its input), where a malicious file could trick the AI into exfiltrating other uploaded files to an attacker's server via image tags. The vulnerability appeared in a recent update but was fixed within 12 days of being reported to Google on February 17, 2024.

Unfurling is when an application automatically expands hyperlinks to show previews, which can be exploited in AI chatbots to leak data. When an attacker uses prompt injection (tricking an AI by hiding instructions in its input) to make the chatbot generate a link containing sensitive information from earlier conversations, the unfurling feature automatically sends that data to a third-party server, potentially exposing private information.

A critical vulnerability was discovered in Qdrant (a vector database system) versions up to 1.6.1, 1.7.4, and 1.8.2 that allows path traversal (a technique where attackers access files outside intended directories) through the Full Snapshot REST API (a web interface for creating system backups). This flaw could let attackers manipulate file paths to access unauthorized files on the system.