AI Sec Watch

President Trump issued an executive order to prevent states from regulating AI by using federal tools like funding withholding and legal challenges, aiming to replace varied state rules with a single federal framework. The order directs federal agencies, including the Attorney General and Commerce Secretary, to challenge state AI laws they view as problematic, while the FTC and FCC will issue guidance on how existing federal laws apply to AI. This action follows a year where ambitious state AI safety proposals, like New York's RAISE Act (which would require AI labs to publish safety practices and report serious incidents), were either weakened or blocked.

Researchers have developed a steganographic method (hiding secret data inside another medium) that embeds hidden messages into compressed neural network models (AI systems made smaller through techniques like quantization, pruning, or distillation). The approach allows a receiver with the correct extraction network to recover the hidden data while ordinary users remain unaware it exists, and the method maintains the model's performance in size, speed, and accuracy.

Researchers proved that spurious local minima (points where a neural network stops improving, but isn't at the best solution) definitely exist in deep CNNs (convolutional neural networks, which are commonly used for image recognition). They created a method to construct these problematic points mathematically and designed a new optimization algorithm (a step-by-step process for improving the network) that can escape from them, showing better accuracy than standard training methods like SGD or Adam on image datasets.

This research addresses backdoor attacks, where poisoned training data (maliciously altered samples inserted into a dataset) causes neural networks to behave incorrectly on specific inputs. The authors propose a defense method called Trap that detects poisoned samples early in training by recognizing they cluster separately from legitimate data, then removes the backdoor by retraining part of the model on relabeled poisoned samples, achieving very high attack detection rates with minimal accuracy loss.

Researchers found that text-to-image diffusion models (AI systems that generate images from text descriptions) can be attacked using backdoors, which are hidden triggers in text that make the model produce unwanted outputs. This paper proposes Dynamic Attention Analysis (DAA), a new detection method that tracks how the model's attention mechanisms (the parts of the AI that focus on relevant information) change over time, since backdoor attacks create different patterns than normal operation. The method achieved strong detection results, correctly identifying backdoored samples about 79% of the time.

Weaviate OSS (open-source software) versions before 1.33.4 have a vulnerability where the fileName field is not properly validated in the transfer logic. An attacker who can call the GetFile method while a shard (a part of a database) is paused and the FileReplicationService (the system that copies files) is accessible could read any files that the service has permission to access.

Weaviate OSS (an open-source vector database) before version 1.33.4 has a path traversal vulnerability (a bug where an attacker can access files outside the intended directory using tricks like ../../..) that allows attackers with database write access to escape the backup restore location and create or overwrite files elsewhere on the system. This could let attackers modify critical files within the application's permissions.

Researchers studied an AI-driven metaverse prototype (a 3D virtual environment enhanced with multi-agent systems, or software that can act independently) designed to train cybersecurity professionals, gathering feedback from 53 experts. The study found that this technology could create personalized, scalable training experiences but identified implementation challenges and proposed six recommendations for organizations considering adopting it.

This research paper addresses a problem in differentially private federated learning (DP-FL, a technique that trains AI models across multiple devices while adding mathematical noise to protect privacy). The paper proposes a new control framework that dynamically adjusts both the amount of noise added and how many communication rounds occur during training, rather than using fixed or randomly adjusted noise levels. Experiments show this approach achieves faster convergence (reaching a good solution quicker) and better accuracy while maintaining the same privacy guarantees.