AI Sec Watch

LLaMA-Factory, a library for customizing large language models, has a vulnerability in versions before 0.9.4 that allows authenticated users to exploit SSRF (server-side request forgery, where the server is tricked into making requests to unintended destinations) and LFI (local file inclusion, where attackers can read files directly from the server) by providing malicious URLs to the chat API. The vulnerability exists because the code doesn't validate URLs before making HTTP requests, allowing attackers to access sensitive internal services or read arbitrary files from the server.

vLLM, a system for running and serving large language models, had a security weakness in how it checked API keys (secret codes that authenticate users) before version 0.11.0rc2. The validation used a basic string comparison that took longer to complete the more correct characters an attacker guessed, allowing them to figure out the key one character at a time through a timing attack (analyzing how long the system takes to respond). This weakness could let attackers bypass authentication and gain unauthorized access.

Federated learning (a way for multiple parties to train an AI model together without sharing their raw data with a central server) normally requires many communication rounds that waste bandwidth and can leak private information. Existing compression methods reduce communication but ignore privacy risks and fail when some clients disconnect. Octopus addresses these issues by using Sketch (a data compression technique) to compress gradients (the direction and size of updates to a model), adding protective masks around the compressed data, and including a strategy to handle disconnected clients.

Federated learning (a training method where multiple parties collaborate to build an AI model without sharing raw data) is vulnerable to model poisoning attacks (where attackers inject harmful updates during training to break the model). This paper proposes MSDFL and HMSDFL, new defensive approaches that strengthen models by improving their stability, meaning they become less sensitive to small changes in their internal parameters, making them more resistant to these poisoning attacks.

The HTMLSectionSplitter class in langchain-text-splitters version 0.3.8 has a vulnerability where it unsafely parses XSLT stylesheets (instructions that transform XML data), allowing attackers to read sensitive files like SSH keys or environment configurations without needing special access. This XXE (XML External Entity, a type of injection attack that exploits how XML parsers handle external files) attack works by default in older versions of the underlying lxml library and can still work in newer versions unless specific security controls are added.

Flowise version 3.0.7 has a file upload vulnerability that lets authenticated users (people with login access) upload any file type without proper checks. Attackers can upload malicious Node.js web shells (programs that let someone run commands on a server remotely), which stay on the server and could lead to RCE (remote code execution, where an attacker runs commands on a system they don't own) if activated through admin mistakes or other vulnerabilities.

SillyTavern, a locally installed interface for interacting with text generation AI models and other AI tools, has a vulnerability in versions before 1.13.4 that allows DNS rebinding (a network attack where an attacker tricks your computer into connecting to a malicious server by manipulating domain name lookups) to let attackers install harmful extensions, steal chat conversations, or create fake login pages. The vulnerability affects the web-based user interface and could be exploited especially when the application is accessed over a local network without SSL (encrypted connections).

Researchers discovered that hyper-parameters (settings that control how a deep reinforcement learning model learns and behaves) can be leaked from closed-box DRL models, meaning attackers can figure out these secret settings just by observing how the model responds to different situations. They created an attack called HyperInfer that successfully inferred hyper-parameters with over 90% accuracy, showing that even restricted AI models may expose information that was meant to stay hidden.

PrivESD is a new system that allows machine learning classification (logistic regression, a technique for categorizing data) to work on encrypted streaming data (continuously flowing information that's been scrambled for privacy) while stored in the cloud. The system splits the computational work between cloud servers and edge devices (computers closer to where data originates) to reduce processing burden and privacy risks, and uses special encryption methods that still allow the system to compare values without revealing the actual data.