Research
Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
2,038 items
Researchers introduce PP-DPIN, a privacy preserving stochastic algorithm for decentralized learning, built on a new mechanism called Positive-Incentive Noise Generator (PING). PING uses network topologies and lightweight encryption-decryption operations to generate correlated noise, aiming to remove the negative effect of privacy noise on convergence while defending against colluding inference attacks. The authors report that at least half of the nodes achieve arbitrarily strong privacy guarantees, and that experiments on computer vision tasks show better performance and attack robustness than state-of-the-art methods.
This paper surveys explainable AI (XAI) for visual recognition models, motivated by deployment in safety-critical areas such as autonomous driving and medical diagnostics. It proposes a multi-dimensional taxonomy based on intent, object, presentation, and methodology, along with evaluation desiderata and metrics, qualitative assessments, and quantitative benchmarks. It also examines the interpretability of Multimodal Large Language Models and practical applications.
This study revisits negative transfer in Unsupervised Domain Adaptation (UDA) through causally disentangled learning. The authors argue that overreliance on non-causal environmental features causes cross-domain discriminative disagreement, which they call environmental disagreement. They propose Reducing Environmental Disagreement (RED), which adversarially trains domain-specific environmental feature extractors to separate domain-invariant causal features from domain-specific non-causal ones, then estimates and reduces the disagreement. Experiments reportedly show RED mitigates negative transfer and achieves state-of-the-art performance.
This study introduces an attribute-based dataset of 31 URL- and similarity-derived features for detecting phishing attacks that target local systems. Five classifiers, including Decision Tree, Random Forest, Logistic Regression, Support Vector Machine and Artificial Neural Network, were compared across three feature-engineered datasets. Logistic Regression performed best at 96.40% accuracy, and 97.48% on a public Kaggle dataset.
This paper formalizes game-theoretic denial-of-service (GDoS) attacks on proof-of-work blockchains, unifying selfish-mining and blockchain-based DoS variants under one definition. It models the problem as a dynamic game with a subgame perfect Nash equilibrium and proposes a cooperative hash-power hopping strategy that requires no protocol changes. Simulations calibrated to Bitcoin hash-power distributions show the strategy cuts attacker revenue gains by more than 20% and prevents throughput degradation across the attack range.
Researchers present the first differential fault attack on GEA-like stream ciphers under the random fault model, applied to the GPRS standard ciphers GEA-1 and GEA-2. Their dedicated algorithm locates the exact fault position, recovering the 64-bit secret keys with time complexities of 2^33.807 for GEA-1 and 2^33.858 for GEA-2. Simulated attacks on ChipWhisperer Lite show both ciphers can be broken within sixteen minutes on a common laptop.
This research addresses cyber deception, the use of traps alongside defense and detection. The authors note that generative AI accelerates the creation of convincing traps, while foundational methods for measuring how effective deception is remain underdeveloped.
This paper proposes the first unified framework for graph matching that handles both partial correspondence, caused by viewpoint occlusions, and noisy correspondence, caused by annotation errors. The method uses a dual-expert design combining Koopmans-Beckmann and Lawler's quadratic assignment formulations (KB-QAP and L-QAP) in an align-fuse-refine pipeline. Experiments on four widely used graph matching datasets show it outperforms 17 competitive baselines in both scenarios.
The Society for Information Management's 45th Annual IT Issues and Trends study surveyed 704 IT executives, including 211 CIOs from 344 unique organizations. AI rose to the top of the most important IT management issues, followed by Cybersecurity and Alignment of IT and Business. Only 54.2% of organizations reported increasing IT headcount, the lowest share since 2020.
MIDAS is a data auditing scheme for mobile devices that outsources computation-intensive auditing metadata generation to edge nodes, leaving mobile devices with only lightweight processing. It embeds edge node identity into the metadata for traceability and adds a proportional load-balanced scheduling algorithm for Multi-MU and Multi-EN scenarios. The authors prove its security by reduction to the discrete logarithm problem in the RSA quadratic residue subgroup under the random oracle model, and real-world experiments show reduced mobile computational overhead and better scalability than existing schemes.
Fix: RED: disentangle each sample into domain-invariant causal features and domain-specific non-causal environmental features by adversarially training domain-specific environmental feature extractors in the opposite domains, then estimate and reduce environmental disagreement based on the domain-specific non-causal environmental features.
IEEE Xplore (Security & AI Journals)OpenAI researchers report that training models to prioritize instructions by trust level, called instruction-hierarchy tasks, improves real-world safety. Models trained this way follow safety specifications in system prompts more closely and resist prompt-injection attacks embedded in tool outputs. The authors also introduce IH-Challenge, a reinforcement learning dataset built to avoid instruction-following shortcuts, unreliable LLM judges, and overrefusals.
Researchers present AGFPS, an automated gradient-free framework that uses evolutionary optimization to steal system prompts from LLM applications. The method treats prompt stealing as a discrete optimization problem, with adversarial queries evolved through elite retention, selection, adaptive crossover and mutation. It reports a 95.2% exact system prompt stealing success rate and outperforms gradient-based methods in 80.6% of scenarios, with queries that transfer across models and remain robust against various defenses.
Researchers study security and privacy risks in multi-LLM collaborative systems, which use structures such as Centralized, Horizontal, and Joint Interaction, and communication methods such as direct, summary, and vote. They propose three attacks, Decision Poisoning Attack, Indirect Echoleak Attack and Information Collision Attack, that exploit inter-LLM interactions for system manipulation and privacy leakage. Experiments show the attacks work across the three structures and three communication methods, and the authors find that malicious instructions must persist and propagate through inter-LLM communication to succeed.
Fix: Cooperative hash-power hopping: when under attack, miners temporarily reallocate hash power to larger pools to preserve expected payoffs and suppress attacker incentives, without any changes to PoW.
IEEE Xplore (Security & AI Journals)Fix: Finally, the possible countermeasures are presented to protect the processed data of massive GPRS devices. (The source text does not specify the countermeasures.)
IEEE Xplore (Security & AI Journals)SeVoAuth is a cloud-based Voiceprint Authentication as a Service (VAaaS) system that stores a synthesized voiceprint rather than the user's real one. During each authentication session, it applies a hash function to transform the synthesized voiceprint's features and generate new verification targets. The authors report an average authentication accuracy of 99.47% and an average Precise Detection Rate (PDR) of 98.35% against various attacks.
Researchers present Distribution Drift Learner (DDL), a black-box attack that bypasses non-transferable learning (NTL) protections using only input-output queries to protected models. DDL reconstructs target domain samples, feeds them to the NTL model, and trains a lightweight autoencoder with distribution perception and reconstruction losses. On Digits, CIFAR10, and STL10, DDL raised target domain accuracy by 81%, from 10%.
A longitudinal study tested the adversarial robustness of GPT, Llama and Qwen model families across successive versions, covering misclassification, jailbreak and hallucination. The authors found that LLM updates do not consistently improve robustness: a later GPT-3.5 version got worse on misclassification and hallucination despite better jailbreak resilience. GPT-4 and GPT-4o showed incrementally higher overall robustness, while larger Llama and Qwen models did not uniformly improve, and larger size did not reliably help.
A banking group introduced a retrieval-augmented, AI-powered compliance assistant, and the article describes what that rollout reveals about future compliance work. It names four tensions in AI-supported compliance: authority illusion, blurred accountability, loss of contextual judgment and widening awareness gaps. It also offers a four-phase framework for moving from reactive AI assistants toward agentic, co-reasoning systems.
ClusterGuard is a secure clustered aggregation scheme for federated learning that aims to protect privacy and resist poisoning attacks. It uses a Verifiable Random Function for fair client clustering, a key-homomorphic masking mechanism with verifiable secret sharing for aggregation within clusters, and a dual filter based on cosine similarity and norm. Experiments on standard datasets report over 2x efficiency gains over advanced secure aggregation methods, and with 20% malicious clients the model keeps accuracy comparable to the original.
Researchers present TPA-VSL, a targeted poisoning attack against vertical split learning, where a model is partitioned between clients and a server. The attack manipulates the embedding model directly, mapping the embedding of a targeted sample to an attacker-chosen class without an obvious trigger pattern. Its two components use diffusion models guided by a multimodal encoder-decoder to mimic the target model, then align the targeted samples' embeddings with those of the desired class. The authors report a 30% higher attack success rate on average than baseline attacks.
PrivateEdit is a privacy-preserving pipeline for face-centric generative image editing. It uses on-device segmentation and masking to separate identity-sensitive facial regions from editable image context, so biometric data are never transmitted to third-party generative models, which need no retraining or access. A tunable masking mechanism lets users choose how much facial information to conceal, and a user interface supports selective anonymization.