AI Sec Watch

This research introduces DeSA, a protocol for secure aggregation (a privacy technique that protects individual data while combining results) in federated learning (a machine learning approach where multiple devices train a shared model without sending raw data to a central server) across decentralized device-to-device networks. The protocol addresses challenges in zero-trust networks (environments where no participant is automatically trusted) by using zero-knowledge proofs (cryptographic methods that verify information is correct without revealing the information itself) to verify model training, protecting against Byzantine attacks (attacks where malicious nodes send false information to disrupt the system), and employing a one-time masking method to maintain privacy while allowing model aggregation.

Subgraph Federated Learning (FL, a system where pieces of a graph are distributed across multiple devices to protect data privacy) is vulnerable to backdoor attacks (hidden malicious functions that cause a model to behave incorrectly when triggered). Researchers developed BEEF, an attack method that uses adversarial perturbations (carefully crafted small changes to input data that fool the model) as hidden triggers while keeping the model's internal parameters unchanged, making the attack harder to detect than existing methods.

vLLM, a system for running and serving large language models, has a Server-Side Request Forgery vulnerability (SSRF, where an attacker tricks a server into making requests to unintended targets) in its multimodal feature before version 0.14.1. The bug exists because two different Python libraries interpret backslashes differently, allowing attackers to bypass security checks and force the vLLM server to send requests to internal network systems, potentially stealing data or causing failures.

PyTorch (a Python package for tensor computation) versions before 2.10.0 have a vulnerability in the `weights_only` unpickler that allows attackers to create malicious checkpoint files (.pth files, which store model data) triggering memory corruption and potentially arbitrary code execution (running attacker-chosen commands) when loaded with `torch.load(..., weights_only=True)`. This is a deserialization vulnerability (a weakness where loading untrusted data can be exploited).

China's DeepSeek AI tool, which caused significant market disruption when it launched a year ago, is now being adopted by an increasing number of US companies. The episode discusses this growing trend of Chinese AI technology being integrated into American business operations.

The White House digitally altered a photograph of an activist's arrest by darkening her skin and distorting her facial features to make her appear more distraught than in the original image posted by the Department of Homeland Security. AI detection tools confirmed the manipulation, raising concerns about how generative AI (systems that create images from text descriptions) and image editing technology can be misused by government to spread false information and reinforce racial stereotypes. The incident highlights the danger of deepfakes (realistic-looking fake media created with AI) and the importance of protecting citizens' right to independently document government actions.

AnythingLLM is an application that lets users feed documents into an LLM so it can reference them during conversations. Versions before 1.10.0 had a security flaw where an API key (QdrantApiKey) for Qdrant, the database that stores document information, could be exposed to anyone without authentication (credentials). If exposed, attackers could read or modify all the documents and knowledge stored in the database, breaking the system's ability to search and retrieve information correctly.

BentoML, a Python library for serving AI models, had a vulnerability (before version 1.4.34) that allowed path traversal attacks (exploiting file path inputs to access files outside intended directories) through its configuration file. An attacker could trick a user into building a malicious configuration that would steal sensitive files like SSH keys or passwords and hide them in the compiled application, potentially exposing them when shared or deployed.

The Kalrav AI Agent plugin for WordPress (versions up to 2.3.3) has a vulnerability in its file upload feature that fails to check what type of file is being uploaded. This allows attackers without user accounts to upload malicious files to the server, potentially leading to RCE (remote code execution, where an attacker can run commands on a system they don't own).