AI & LLM Vulnerabilities

LMDeploy (a tool for deploying large language models) has a critical vulnerability in how it loads model configurations from HuggingFace. When loading a model, the software uses eval() (a Python function that executes code from text) on an untrusted value called quant_dtype without checking if it's safe. An attacker can publish a malicious model on HuggingFace with crafted code hidden in the quant_dtype field, allowing them to run arbitrary commands on any computer that loads the model using LMDeploy.

Kedro-Datasets (a tool that connects data sources to Kedro, a framework for building data pipelines) had a vulnerability in versions 5.0.0 through 9.5.0 where its PyTorch model loader didn't safely load .pt files, allowing attackers to run arbitrary code (RCE, remote code execution) if someone loads a malicious model file from an untrusted source on PyTorch versions before 2.6. This only affected users of the optional experimental component and only when loading untrusted files.

Stanza is a Python library from Stanford for processing natural language (breaking text into words, sentences, identifying named entities, and analyzing grammar structure). Before version 1.14.0, it had a security flaw where it extracted downloaded files without checking if they tried to escape their intended folder, allowing a malicious file to overwrite important system files and potentially run harmful code.

ESPnet (a speech processing AI framework) before version 202609 has a vulnerability where it loads pretrained model checkpoints (saved AI model files) using an unsafe method that can execute hidden malicious code. An attacker can create a fake checkpoint file that runs whatever code they want when someone tries to use that file to initialize or improve an AI model.

vLLM (a tool for running large language models efficiently) versions before 0.28.0 have a remote code execution vulnerability (a flaw that lets attackers run their own code on a system) in the LlavaOnevision2 processor loader. Even when a user sets trust_remote_code to False (a safety setting meant to prevent loading untrusted code), the vulnerability ignores this setting and allows attackers to execute malicious code by hiding it in a model file.

IBM Langflow OSS (open-source software) versions 1.0.0 through 1.11.2 has a vulnerability where an authenticated attacker (someone with valid login credentials) can delete any files or folders on the server because the software doesn't properly restrict which directories users can access. This happens because the application fails to limit file paths (the addresses of files on disk) to only safe, intended locations.

LLaMA-Factory, an AI model tool, has a security flaw where its API (application programming interface) that mimics OpenAI's system fails to properly block requests to internal servers. Attackers can bypass this protection using HTTP redirects (where a website sends you to another location) or DNS rebinding (tricking the system into looking up a different address) to access sensitive internal data and cloud metadata endpoints (services that store configuration information).

A vulnerability in Hugging Face Transformers (a library for building AI models) versions 4.49.0 through 5.8.1 allows attackers to write Python files to a user's computer without permission when using the GenerativePreTrainedModel.load_custom_generate() function. The problem occurs because the function downloads and saves remote code before checking if the user trusts that code, breaking the security model used elsewhere in the library. Even if a user refuses the trust prompt, malicious code files are already saved to disk and can persist across sessions.

vLLM (a large language model serving framework) versions up to 0.17.0 have a vulnerability where two functions that fetch media files from user-provided URLs do not limit how much data they download, allowing attackers to crash the server by pointing it to extremely large files and exhausting its memory (a DoS or denial of service attack, where a system becomes unusable).

whichllm (a tool for working with LLMs) versions before 0.5.16 have a code injection vulnerability in the run and snippet commands that lets a remote attacker execute arbitrary code (running commands they choose on your computer). The vulnerability exists because the tool takes filenames from HuggingFace (a model repository) and puts them directly into Python code without checking them first, so a specially crafted filename with special characters can break out and run malicious code before the model is even downloaded.

Vocos, an AI model loading library, has a security flaw where it loads and runs any code specified in a configuration file without checking if that code is safe (instantiate_class, a function that creates objects based on config file instructions, doesn't use an allowlist to restrict which classes can be loaded). This means if you load a model from an untrusted source, the attacker who controls that source can run arbitrary code on your computer during the loading process.

Nokogiri versions before 1.19.3 have a ReDoS vulnerability (regular expression denial of service, where carefully crafted input causes a regex pattern to take exponentially longer to process) in how it parses CSS selectors. Attackers can exploit this by injecting malicious CSS selectors into methods like Node#css and Node#at_css to make the application hang or crash.

mcp-shell has a security flaw where its default configuration allows `/bin/bash` as an allowed command, but the validator only checks the first word of a command and ignores flags like `-c`. This means an attacker can send `/bin/bash -c <any-command>` to bypass the allowlist (a restrictions list) and run any command on the system, such as `id` or `curl`, with no authentication needed.

Xinference (a model-serving tool) was loading AI models from Hugging Face with remote code execution (the ability to run code from external sources) always enabled before version 2.12.0, and users had no way to turn it off. An attacker who could register a new model could trick the system into running malicious code hidden in the model's configuration files, giving that code the same permissions as the server running Xinference.