AI Sec Watch

vLLM, a system for running and serving large language models, had a security weakness in how it checked API keys (secret codes that authenticate users) before version 0.11.0rc2. The validation used a basic string comparison that took longer to complete the more correct characters an attacker guessed, allowing them to figure out the key one character at a time through a timing attack (analyzing how long the system takes to respond). This weakness could let attackers bypass authentication and gain unauthorized access.

Federated learning (a way for multiple parties to train an AI model together without sharing their raw data with a central server) normally requires many communication rounds that waste bandwidth and can leak private information. Existing compression methods reduce communication but ignore privacy risks and fail when some clients disconnect. Octopus addresses these issues by using Sketch (a data compression technique) to compress gradients (the direction and size of updates to a model), adding protective masks around the compressed data, and including a strategy to handle disconnected clients.

Federated learning (a training method where multiple parties collaborate to build an AI model without sharing raw data) is vulnerable to model poisoning attacks (where attackers inject harmful updates during training to break the model). This paper proposes MSDFL and HMSDFL, new defensive approaches that strengthen models by improving their stability, meaning they become less sensitive to small changes in their internal parameters, making them more resistant to these poisoning attacks.

The HTMLSectionSplitter class in langchain-text-splitters version 0.3.8 has a vulnerability where it unsafely parses XSLT stylesheets (instructions that transform XML data), allowing attackers to read sensitive files like SSH keys or environment configurations without needing special access. This XXE (XML External Entity, a type of injection attack that exploits how XML parsers handle external files) attack works by default in older versions of the underlying lxml library and can still work in newer versions unless specific security controls are added.

Flowise version 3.0.7 has a file upload vulnerability that lets authenticated users (people with login access) upload any file type without proper checks. Attackers can upload malicious Node.js web shells (programs that let someone run commands on a server remotely), which stay on the server and could lead to RCE (remote code execution, where an attacker runs commands on a system they don't own) if activated through admin mistakes or other vulnerabilities.

SillyTavern, a locally installed interface for interacting with text generation AI models and other AI tools, has a vulnerability in versions before 1.13.4 that allows DNS rebinding (a network attack where an attacker tricks your computer into connecting to a malicious server by manipulating domain name lookups) to let attackers install harmful extensions, steal chat conversations, or create fake login pages. The vulnerability affects the web-based user interface and could be exploited especially when the application is accessed over a local network without SSL (encrypted connections).

Researchers discovered that hyper-parameters (settings that control how a deep reinforcement learning model learns and behaves) can be leaked from closed-box DRL models, meaning attackers can figure out these secret settings just by observing how the model responds to different situations. They created an attack called HyperInfer that successfully inferred hyper-parameters with over 90% accuracy, showing that even restricted AI models may expose information that was meant to stay hidden.

PrivESD is a new system that allows machine learning classification (logistic regression, a technique for categorizing data) to work on encrypted streaming data (continuously flowing information that's been scrambled for privacy) while stored in the cloud. The system splits the computational work between cloud servers and edge devices (computers closer to where data originates) to reduce processing burden and privacy risks, and uses special encryption methods that still allow the system to compare values without revealing the actual data.

Hard sample mining (HSM, a technique for selecting the most difficult training examples to focus a model's learning) has emerged as a method to improve how efficiently deep neural networks (AI systems based on interconnected layers inspired by brain neurons) train and make them more robust to errors. This survey article reviews different HSM approaches and explains how they help address training inefficiency and data distribution biases (when training data doesn't represent real-world scenarios fairly) in deep learning.