AI Sec Watch

Large language models face four main types of adversarial threats: privacy breaches (exposing sensitive data the model learned), integrity compromises (corrupting the model's outputs or training data), adversarial misuse (using the model for harmful purposes), and availability disruptions (making the model unavailable or slow). The article organizes these threats by their attackers' goals to help understand the landscape of vulnerabilities in LLMs.

This short story examines privacy risks that arise when companies are bought and sold, particularly concerning AI digital twins (AI models that replicate a specific person's behavior and knowledge) and the problems that occur when organizations fail to threat model (identify and plan for potential security risks in) major changes to their systems and technology. The story raises ethical questions about these scenarios.

CA-CI is a framework that combines two concepts—contextual integrity (the idea that information flows should match their specific social context) and the capabilities approach (a method for evaluating human dignity and well-being)—to help govern AI systems responsibly. The framework aims to operationalize (put into practical use) the EU AI Act's requirements for assessing fundamental rights impacts, setting harm thresholds, and planning ahead for potential AI risks.

Vision-Language Models (VLMs, AI systems that understand both images and text together) like CLIP are powerful but vulnerable to adversarial attacks (malicious inputs designed to fool AI systems, especially in images). This research presents NAP-Tuning, a method that uses learnable text prompts and lightweight neural modules called TokenRefiners to clean up distorted features inside the model's layers, making these systems more resistant to such attacks while keeping normal performance intact.

Researchers discovered a jailbreak technique called semantic chaining that tricks certain LLMs (AI models trained on massive amounts of text) by breaking malicious requests into small, separate chunks that the model processes without understanding the overall harmful intent. This vulnerability affected models like Gemini Nano and Grok 4, which failed to recognize the dangerous purpose when instructions were split across multiple parts.

Journalists highlight three major cybersecurity priorities: fixing known weaknesses in software, getting ready for quantum computing threats (powerful computers that could break current encryption), and improving how AI systems are built and used. The piece emphasizes that the cybersecurity industry needs to focus on these areas to stay ahead of emerging risks.

Concept drift (when data patterns change over time due to evolving attacks or environments) is a major problem for machine learning models used in cybersecurity, since frequent retraining is expensive and hard to understand. DriftTrace is a new system that detects concept drift at the sample level (individual data points) using a contrastive learning-based autoencoder (a type of neural network that learns patterns without needing lots of labeled examples), explains which features caused the drift using feature selection, and adapts to drift by balancing training data. The system was tested on malware and network intrusion datasets and achieved strong results, outperforming existing approaches.

Federated learning (collaborative model training where participants share only gradients, not raw data) is vulnerable to gradient inversion attacks, where adversaries reconstruct sensitive training data from the shared gradients. The paper proposes Gradient Dropout, a defense that randomly scales some gradient components and replaces others with Gaussian noise (random numerical values) to disrupt reconstruction attempts while maintaining model accuracy.

This research introduces SEGA, a method for attacking No-Reference Image Quality Assessment models (AI systems that evaluate image quality without comparing to a reference image) in black-box scenarios where attackers cannot see the target model's code. SEGA works by using Gaussian smoothing (a mathematical technique that approximates gradients, or the direction of change in the model) across multiple source models and applying a filter to make attacks harder to detect. The method successfully demonstrates improved ability to transfer attacks across different NR-IQA models.