AI Sec Watch

LlamaIndex versions up to 0.11.6 contain a vulnerability where the BGEM3Index.load_from_disk() function uses pickle.load() (a Python method that converts stored data back into objects) to read files from a user-provided directory without checking if they're safe. An attacker could provide a malicious pickle file that executes arbitrary code (runs any commands they want) when a victim loads the index from disk.

LibreChat, a ChatGPT clone with extra features, has a vulnerability in versions before v0.8.2-rc2 where its MCP stdio transport (a communication method for connecting components) accepts commands without checking if they're safe, letting any logged-in user run shell commands as root inside a container with just one API request. This is a serious authorization flaw because it bypasses permission checks.

OpenCode, an open source AI coding agent, has a vulnerability in its markdown renderer that allows arbitrary HTML to be inserted into the web interface without proper sanitization (blocking of malicious code). Because there is no protection like DOMPurify (a tool that removes dangerous HTML) or CSP (content security policy, rules that restrict what code can run), an attacker who controls what the AI outputs could execute JavaScript (code that runs in the browser) on the local web interface.

OpenCode is an open source AI coding agent that, before version 1.0.216, automatically started an unauthenticated HTTP server (a service that accepts web requests without requiring a password or login). This allowed any local process or website with permissive CORS (a web setting that controls which websites can access a server) to execute arbitrary shell commands with the user's privileges, meaning someone could run malicious commands on the affected computer.

MLFlow versions up to 3.4.0 have a vulnerability where the REST server (the interface that external programs use to communicate with MLFlow) doesn't properly validate Origin headers, which are security checks that prevent unauthorized websites from making requests. This allows attackers to use DNS rebinding attacks (tricks where malicious websites disguise their identity to bypass security protections) to query, modify, or delete experiments, potentially stealing or destroying data.

Unlearnable examples are protective noises added to private data to prevent AI models from learning useful information from them, but this paper shows that data augmentation (a common technique that creates variations of training data to improve model performance) can undo this protection and restore learnability from 21.3% to 66.1% accuracy. The researchers propose Armor, a defense framework that adds protective noise while accounting for data augmentation effects, using a surrogate model (a practice model used to simulate the real training process) and smart augmentation selection to keep private data unlearnable even after augmentation is applied.

This research addresses how to identify whether one machine learning model is derived from another model through modification techniques (adjusting or fine-tuning an existing model rather than training from scratch), and how to measure how much two models differ from each other. The authors propose a method that determines lineage (derivative relationships) by checking if two models' parameters exist in the same local optimum of the loss landscape (the mathematical space of possible model configurations), and measure closeness by analyzing how their decision boundaries (the lines or surfaces that separate different predictions) differ from each other.

vLLM is a serving engine for running large language models, and versions 0.6.4 through 0.11.x have a vulnerability where attackers can crash the server by sending a tiny 1x1 pixel image to models using the Idefics3 vision component, causing a dimension mismatch (a size incompatibility between data structures) that terminates the entire service.

The BetterDocs plugin for WordPress (all versions up to 4.3.3) has a vulnerability that exposes sensitive information, allowing authenticated attackers with contributor-level access or higher to extract data including OpenAI API keys stored in the plugin settings through the scripts() function. This affects any WordPress site using the plugin where users have contributor-level permissions or above.