Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
Vision-Language Models (VLMs, AI systems that understand both images and text together) like CLIP are powerful but vulnerable to adversarial attacks (malicious inputs designed to fool AI systems, especially in images). This research presents NAP-Tuning, a method that uses learnable text prompts and lightweight neural modules called TokenRefiners to clean up distorted features inside the model's layers, making these systems more resistant to such attacks while keeping normal performance intact.
This research addresses authentication risks in wireless IoT devices by proposing LiteNP-Net, a lightweight neural network for physical layer authentication (PLA, a security method that verifies device identity using unique wireless channel characteristics). The approach combines hypothesis testing theory with deep learning to create a system that works effectively even without detailed prior knowledge of wireless channel properties, and testing showed it performs better than existing methods in real-world Wi-Fi environments.
Concept drift (when data patterns change over time due to evolving attacks or environments) is a major problem for machine learning models used in cybersecurity, since frequent retraining is expensive and hard to understand. DriftTrace is a new system that detects concept drift at the sample level (individual data points) using a contrastive learning-based autoencoder (a type of neural network that learns patterns without needing lots of labeled examples), explains which features caused the drift using feature selection, and adapts to drift by balancing training data. The system was tested on malware and network intrusion datasets and achieved strong results, outperforming existing approaches.
Fix: DriftTrace addresses concept drift through three mechanisms: (1) detecting drift at the sample level using a contrastive learning-based autoencoder without requiring extensive labeling, (2) employing a greedy feature selection strategy to explain which input features are relevant to drift detection decisions, and (3) leveraging sample interpolation techniques to handle data imbalance during adaptation to the drift.
IEEE Xplore (Security & AI Journals)Federated learning (collaborative model training where participants share only gradients, not raw data) is vulnerable to gradient inversion attacks, where adversaries reconstruct sensitive training data from the shared gradients. The paper proposes Gradient Dropout, a defense that randomly scales some gradient components and replaces others with Gaussian noise (random numerical values) to disrupt reconstruction attempts while maintaining model accuracy.
Fix: Gradient Dropout is applied as a defense mechanism: it perturbs gradients by randomly scaling a subset of components and replacing the remainder with Gaussian noise, applied across all layers of the model. According to the source, this approach yields less than 2% accuracy reduction relative to baseline while significantly impeding reconstruction attacks.
IEEE Xplore (Security & AI Journals)This research introduces SEGA, a method for attacking No-Reference Image Quality Assessment models (AI systems that evaluate image quality without comparing to a reference image) in black-box scenarios where attackers cannot see the target model's code. SEGA works by using Gaussian smoothing (a mathematical technique that approximates gradients, or the direction of change in the model) across multiple source models and applying a filter to make attacks harder to detect. The method successfully demonstrates improved ability to transfer attacks across different NR-IQA models.
ZUMA is a training-free framework for multimodal anomaly detection (MAD, identifying unusual patterns using both image and 3D data together) that works without needing labeled training examples, addressing privacy concerns. It uses CLIP (a model trained on images and text) and introduces cross-domain calibration (a technique that bridges differences between how CLIP was trained and how 3D point cloud data works) and dynamic semantic interaction (using natural language descriptions as reference points to spot anomalies) to detect defects in 2D images, 3D objects, or both together without requiring training.
This research addresses the risk that AI models trained on private data could leak sensitive information if attackers extract data from them. The authors propose a method called differentially private synthetic distillation, which converts a trained model into a privacy-protected version without needing access to the original private data, using a generator to create synthetic data and noise to obscure sensitive patterns.
Subgraph Federated Learning (FL, a system where pieces of a graph are distributed across multiple devices to protect data privacy) is vulnerable to backdoor attacks (hidden malicious functions that cause a model to behave incorrectly when triggered). Researchers developed BEEF, an attack method that uses adversarial perturbations (carefully crafted small changes to input data that fool the model) as hidden triggers while keeping the model's internal parameters unchanged, making the attack harder to detect than existing methods.
This research introduces DeSA, a protocol for secure aggregation (a privacy technique that protects individual data while combining results) in federated learning (a machine learning approach where multiple devices train a shared model without sending raw data to a central server) across decentralized device-to-device networks. The protocol addresses challenges in zero-trust networks (environments where no participant is automatically trusted) by using zero-knowledge proofs (cryptographic methods that verify information is correct without revealing the information itself) to verify model training, protecting against Byzantine attacks (attacks where malicious nodes send false information to disrupt the system), and employing a one-time masking method to maintain privacy while allowing model aggregation.
Researchers discovered that speech translation (ST) systems, which convert spoken words from one language to another, can be tricked by specially crafted audio manipulations that are imperceptible to human ears. They demonstrated two attack methods: adapting techniques from ASR (automatic speech recognition) attacks and using music-based perturbations to guide the system toward producing harmful outputs. These attacks worked across multiple languages and models, revealing a fundamental weakness in how current speech translation systems process and understand audio.
This paper introduces ATRNet-STAR, a new large dataset for training AI models to recognize objects in Synthetic Aperture Radar (SAR) images, which are pictures taken using microwave radar rather than cameras. The dataset contains over 190,000 labeled images of 40 different vehicle types collected under realistic conditions, making it about 10 times larger than the previous standard dataset (MSTAR from the 1990s). The authors evaluated 15 different AI methods on this dataset to show how well current techniques work and identify directions for future research.
Adversarial imitation learning (AIL, a technique where an AI learns to mimic expert behavior by competing against a discriminator network) has worked well in practice but lacked solid theoretical foundations except in oversimplified settings. This paper introduces OPT-AIL (optimization-based adversarial imitation learning), a new framework that works with general function approximation (flexible neural network models rather than simple lookup tables), and proves it can learn expert-level policies efficiently while remaining practical to implement.
This research addresses multi-object navigation (MON), where an AI agent must find multiple targets in unknown environments by balancing immediate actions with long-term planning. Current methods focus too much on local path optimization, causing slow learning and getting stuck in trap states. The researchers propose GDWO (Goal-oriented Dynamic Weight Optimization), an algorithm that dynamically adjusts how much each target task contributes to the overall optimization by using gradient-based updates (mathematical techniques that improve decisions step-by-step) and normalizing weights based on navigation success rates, which improves learning efficiency and path planning.
Mobile super apps (large platforms that host smaller third-party applications, called miniapps, which share the same underlying services) create new security risks because multiple apps can access shared resources and data. Researchers studied how these ecosystems work, identified security vulnerabilities and potential abuses, and developed recommendations to make super app platforms safer while keeping them easy to use.
This research analyzes how discussions about Generative AI spread across different industries (like media, healthcare, and finance) in the six months after ChatGPT's release, using social media data and innovation theory. The study found that different industries had different concerns: media and marketing focused on content generation with positive views, while healthcare and finance were more cautious and focused on analysis. Misinformation was the biggest concern overall, and the research showed that emotional reactions (sentiment) were the main factor driving how quickly information about AI spread between people.
Generative artificial intelligence (GAI, AI systems that create new text, images, or code) is significantly changing how information systems are taught in universities. IS educators are discussing both the benefits and risks of GAI, including concerns about academic integrity (students using AI to cheat), and they are developing recommendations for how to responsibly teach with and about GAI in the classroom.
The article demonstrates how attackers can use crafted prompts to trick AI assistants into running harmful database queries through prompt-to-SQL injection attacks (where malicious instructions hidden in user input cause an AI to generate dangerous database commands). It identifies vulnerabilities in real systems and describes practical defenses including query filtering, rewriting, data preloading, and using another AI model as a security guard.
Fix: The source mentions four explicit defenses: query filtering, query rewriting, data preloading, and large-language-model-based guards (using another AI model to validate or block dangerous queries).
IEEE Xplore (Security & AI Journals)This research paper analyzes how companies that invest in digital technologies, including AI, affect their greenhouse gas emissions and natural resource use. The study found that companies investing in these technologies tend to reduce their emissions and consume fewer natural resources, suggesting that digital tools can help address environmental challenges.
This paper addresses white-box attacks (scenarios where attackers can see all the inner workings of an encryption system and control the computer it runs on), which are harder to defend against than black-box attacks (where attackers cannot see the implementation). The authors propose a new method to protect symmetric encryption algorithms that use substitution-permutation networks (a common encryption structure that substitutes and rearranges data) by adding secret components to lookup tables, making the encryption stronger without changing the final encrypted message.
This research addresses how to remove unwanted information from pre-trained vision models (AI systems trained to understand images) when users or model owners request it, especially when these deletion requests come one after another over time. The researchers propose Group Sparse LoRA (GS-LoRA), a technique that uses Low-Rank Adaptation modules (efficient add-on components that modify specific neural network layers) to selectively forget targeted classes or information while keeping the rest of the model working well, even when some training data is missing.
Fix: The paper proposes two explicit solutions: (1) Group Sparse LoRA (GS-LoRA), which uses Low-Rank Adaptation modules to fine-tune Feed-Forward Network layers in Transformer blocks for each forgetting task independently, combined with group sparse regularization to automatically select and zero out specific LoRA groups. (2) GS-LoRA++, an extension that incorporates prototype information as additional supervision, moving logits (output scores) away from the original prototype of forgotten classes while pulling logits closer to prototypes of remaining classes.
IEEE Xplore (Security & AI Journals)