AI Sec Watch

ZUMA is a training-free framework for multimodal anomaly detection (MAD, identifying unusual patterns using both image and 3D data together) that works without needing labeled training examples, addressing privacy concerns. It uses CLIP (a model trained on images and text) and introduces cross-domain calibration (a technique that bridges differences between how CLIP was trained and how 3D point cloud data works) and dynamic semantic interaction (using natural language descriptions as reference points to spot anomalies) to detect defects in 2D images, 3D objects, or both together without requiring training.

This research introduces DeSA, a protocol for secure aggregation (a privacy technique that protects individual data while combining results) in federated learning (a machine learning approach where multiple devices train a shared model without sending raw data to a central server) across decentralized device-to-device networks. The protocol addresses challenges in zero-trust networks (environments where no participant is automatically trusted) by using zero-knowledge proofs (cryptographic methods that verify information is correct without revealing the information itself) to verify model training, protecting against Byzantine attacks (attacks where malicious nodes send false information to disrupt the system), and employing a one-time masking method to maintain privacy while allowing model aggregation.

Subgraph Federated Learning (FL, a system where pieces of a graph are distributed across multiple devices to protect data privacy) is vulnerable to backdoor attacks (hidden malicious functions that cause a model to behave incorrectly when triggered). Researchers developed BEEF, an attack method that uses adversarial perturbations (carefully crafted small changes to input data that fool the model) as hidden triggers while keeping the model's internal parameters unchanged, making the attack harder to detect than existing methods.

When multiple AI agents work together, they can amplify hallucinations (false or made-up information generated by LLMs), creating a 'snowballing effect' where errors get worse as agents collaborate. This paper proposes a framework that uses semantic reasoning (understanding meaning and logical relationships) to detect and reduce hallucination spread in multiagent systems without changing how the underlying models work.

Researchers discovered that speech translation (ST) systems, which convert spoken words from one language to another, can be tricked by specially crafted audio manipulations that are imperceptible to human ears. They demonstrated two attack methods: adapting techniques from ASR (automatic speech recognition) attacks and using music-based perturbations to guide the system toward producing harmful outputs. These attacks worked across multiple languages and models, revealing a fundamental weakness in how current speech translation systems process and understand audio.

vLLM, a system for running and serving large language models, has a Server-Side Request Forgery vulnerability (SSRF, where an attacker tricks a server into making requests to unintended targets) in its multimodal feature before version 0.14.1. The bug exists because two different Python libraries interpret backslashes differently, allowing attackers to bypass security checks and force the vLLM server to send requests to internal network systems, potentially stealing data or causing failures.

PyTorch (a Python package for tensor computation) versions before 2.10.0 have a vulnerability in the `weights_only` unpickler that allows attackers to create malicious checkpoint files (.pth files, which store model data) triggering memory corruption and potentially arbitrary code execution (running attacker-chosen commands) when loaded with `torch.load(..., weights_only=True)`. This is a deserialization vulnerability (a weakness where loading untrusted data can be exploited).

China's DeepSeek AI tool, which caused significant market disruption when it launched a year ago, is now being adopted by an increasing number of US companies. The episode discusses this growing trend of Chinese AI technology being integrated into American business operations.

The White House digitally altered a photograph of an activist's arrest by darkening her skin and distorting her facial features to make her appear more distraught than in the original image posted by the Department of Homeland Security. AI detection tools confirmed the manipulation, raising concerns about how generative AI (systems that create images from text descriptions) and image editing technology can be misused by government to spread false information and reinforce racial stereotypes. The incident highlights the danger of deepfakes (realistic-looking fake media created with AI) and the importance of protecting citizens' right to independently document government actions.