AI Sec Watch

This research proposes a new method called DP-QAM (Differentially Private Quadrature Amplitude Modulation) to solve privacy and communication problems in federated analytics (a system where multiple devices analyze data together without sending raw data to a central server). The method takes advantage of natural errors that occur during data compression and wireless transmission to add extra privacy protection, while balancing privacy, communication efficiency, and accuracy.

Researchers discovered a new attack called Lure that targets generative language models (GLMs, which are AI systems that generate text) during the fine-tuning process (when developers customize an open-source model with their own data). By hiding malicious code in the source code of an open-source model, attackers can trick a fine-tuned model into remembering and later revealing the proprietary data used to customize it through specially crafted prompts (input text designed to trigger specific outputs).

QuEST is a new framework that makes backdoor attacks (hidden malicious behaviors injected into AI models) more stealthy and efficient when models undergo quantization (compressing models to use less memory and computation). The framework uses special training techniques and parameter sharing to hide the attack from detection systems while reducing the computational resources needed to carry out the attack.

This research addresses vulnerabilities in Federated Learning (FL, a system where multiple computers train an AI model together without sharing their raw data), which faces attacks from malicious participants and privacy leaks from gradient updates (the numerical adjustments that improve the model). The authors propose a new method combining homomorphic encryption (a way to perform calculations on encrypted data without decrypting it) and dimension compression (reducing the size of data while keeping important relationships intact) to protect privacy and defend against Byzantine attacks (when malicious actors send corrupted data to sabotage the system) while reducing computational costs by 25 to 35 times.

Large vision-language models (LVLMs, which are AIs that understand both images and text) can be attacked using simple visual transformations, such as rotations or color changes, that fool them into giving wrong answers. Researchers found that combining multiple harmful transformations can make these attacks more effective, and they can be optimized using gradient approximation (a mathematical technique to find the best attack parameters). This research highlights a previously overlooked safety risk in how well LVLMs resist these kinds of adversarial attacks (attempts to trick AI systems).

This research addresses security challenges in Internet of Things (IoT) devices by improving radio frequency fingerprint identification (RFFI, a method that uniquely identifies devices based on their wireless signal characteristics) using federated learning (a distributed AI training approach where data stays on local devices rather than being sent to a central server). The paper proposes a feature alignment strategy to handle non-IID data (data that isn't uniformly distributed across different receivers), which occurs when different receivers have different hardware and environmental conditions, and demonstrates that the approach achieves 90.83% identification accuracy with improved stability compared to existing federated learning methods.

AdaParse is a framework that can identify the specific settings (hyperparameters, which are configuration values that control how a model behaves) used to create AI-generated images by analyzing those images in detail. Unlike older methods that use a single general fingerprint (a characteristic pattern), AdaParse creates customized fingerprints for each image, allowing it to distinguish between images made with different settings across many different generative models (AI systems that create images).

Anthropic's CEO is negotiating with the U.S. Department of Defense to repair their relationship after talks broke down over the Pentagon's demand for unrestricted access to Anthropic's AI system. The military had labeled Anthropic a 'supply chain risk' (a concern that a vendor could compromise national security), and competitors like OpenAI are now pursuing defense contracts in Anthropic's absence.

A group of 30 former defense and intelligence officials sent a letter to Congress opposing the Pentagon's decision to designate Anthropic a supply chain risk (a classification normally used to block foreign threats from infiltrating U.S. systems). The group argues this decision weakens U.S. competitiveness in AI and sets a dangerous precedent by penalizing an American company for refusing to remove safeguards against mass surveillance and autonomous weapons.