Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
Voice authentication on smartphones is vulnerable to spoofing attacks, where attackers replay recorded voice samples through loudspeakers to trick the system. MagLive is a new security method that detects whether a voice is from a real person or a loudspeaker by analyzing magnetic pattern changes (detected by the smartphone's built-in magnetometer) using a machine learning model called TF-CNN-SAF (a type of neural network designed to extract useful patterns from data).
This research proposes TCroS, a system for securely sharing sensitive data across different manufacturer domains in Internet of Vehicles (IoV, connected vehicles that communicate over the internet) while allowing users to search for specific information using keywords. The system uses proxy re-encryption (a technique that transforms encrypted data so it can be decrypted by different authorized parties) and embeds requester identities into decryption keys so that if an encryption key is leaked, the source can be traced. The authors also created an extended version called TCroSS that adds privacy-preserving keyword search with Boolean queries (AND, OR, NOT operations) to find authorized data efficiently while protecting against keyword guessing attacks.
FreqTransNet is a new deep learning watermarking model that combines convolutional modules, Transformer structures (neural networks that use self-attention to understand relationships between distant parts of data), and frequency-domain transformations (mathematical techniques that analyze images by breaking them into component frequencies) to embed invisible marks into images more robustly. The model outperforms existing watermarking methods, achieving better visual quality and maintaining over 97% accuracy in extracting watermarks even when images are attacked or modified.
A Decentralized Randomness Beacon (DRB, a cryptographic tool that generates unpredictable, publicly verifiable randomness for distributed systems) is used in blockchain and distributed networks, but existing protocols like GRandLine and RandFlash expose participants' identities during leader election, risking privacy leaks. The paper proposes PADRE, a new privacy-aware DRB protocol that hides participant identities while maintaining security and efficiency, using a new cryptographic primitive called anonymous threshold verifiable random functions (ATVRF).
NiIas is a security protocol designed for Multi-access Edge Computing (MEC, a system where computing resources are placed closer to users at the network edge) that allows devices to authenticate and send data immediately without the usual setup delays required by traditional protocols. The protocol uses identity-based cryptography (a method where users' public keys are derived from their identity rather than certificates) and an authenticate-before-decryption mechanism to filter out unauthorized traffic and protect against denial-of-service attacks (attempts to overwhelm a system with traffic to make it unavailable).
This research paper presents RASE, a new method for collecting data from Internet of Things devices (connected sensors that gather information) while protecting people's privacy from disclosure attacks (attempts to figure out what specific individuals' data is). RASE works in three steps: first adding noise (random false data) to real readings, then randomly scrambling who sent what data so senders can't be linked to receivers, and finally calculating approximate totals from the scrambled data.
Researchers tested code generated by five different LLM (large language model, AI systems that write text) families across four programming languages and found significant security weaknesses, especially in C and C++ where memory safety issues (bugs that let attackers access or corrupt memory) and hard-coded secrets (passwords or keys written directly in code) were common. The study revealed that LLMs often fail to use modern security features available in newer versions of programming tools and tend to rely on outdated, less secure methods. The researchers conclude that LLMs need to be improved to generate code that follows current security best practices.
This research identifies how attackers can exploit non-control data in the Linux kernel's file system (the part of the OS that manages files and folders) to gain unauthorized access, even when control-flow integrity (a security technique that prevents attackers from hijacking program execution paths) is in place. The study developed a framework to automatically find vulnerable data objects and demonstrated 10 working attacks against the kernel with modern security protections enabled.
This article addresses the problem of label noise (incorrect training data labels) in graph neural networks (GNNs, which are AI models that learn from networks of connected data points). GNNs are especially vulnerable to label noise because they share information across connected nodes, which can amplify mistakes. The authors propose NomiGNN, a new framework that estimates noise patterns in the training data and adjusts the learning process to reduce noise's negative effects.
Fix: The source describes NomiGNN's approach: it 'estimates noise distributions and refines loss optimization to counteract noise during training,' incorporates 'edge labels for a fresh prediction task' to learn relationships between samples, includes 'pseudoedge labeling and iterative learning to remedy label shortages and estimation inaccuracies.' The authors demonstrate NomiGNN's effectiveness through experiments on real-world graphs, showing it 'outperform[s] eight benchmark GNN models' in robustness against noisy labels.
IEEE Xplore (Security & AI Journals)This research paper describes a method for estimating the state (current condition) of complex networks that have unknown nonlinearities (behaviors that don't follow simple linear patterns) and sensors powered by energy harvesting (collecting energy from their environment). The approach uses neural networks (machine learning models inspired by how brains process information) to approximate these unknown nonlinearities, and it only allows sensors to transmit data when they have enough stored energy to do so, helping conserve power in resource-limited systems.
This research paper addresses cross-domain few-shot semantic segmentation (CD-FSS), which is the problem of training AI models to identify and outline objects in images across different visual domains (like photos vs. drawings) using very limited labeled data. The authors propose a new method called FOYF that uses multiple specialized attention mechanisms (parts of the model that focus on different types of information) based on Swin Transformer (a type of neural network architecture) to better capture details within and between objects, and they show their approach performs better on benchmark tests than existing methods.
This research addresses negative transfer, which occurs when an AI model performs worse after trying to apply knowledge from one domain (a labeled dataset) to a different domain (an unlabeled dataset) due to significant differences between them. The study identifies that models relying too heavily on non-causal environmental features (irrelevant details that don't actually cause predictions) creates disagreement across domains, harming performance. The proposed solution, called RED (Reducing Environmental Disagreement), separates each sample into causal features (the truly relevant information) and non-causal environmental features, then reduces the disagreement between domains based on these environmental features.
Fix: The proposed solution is RED (Reducing Environmental Disagreement), which "disentangles each sample into domain-invariant causal features and domain-specific non-causal environmental features via adversarially training domain-specific environmental feature extractors in the opposite domains. Subsequently, RED estimates and reduces environmental disagreement based on domain-specific non-causal environmental features."
IEEE Xplore (Security & AI Journals)This paper surveys eXplainable AI (XAI, methods for making AI decisions understandable to humans) in visual recognition, which is increasingly important for safety-critical applications like autonomous driving and medical diagnostics. The survey organizes XAI approaches by intent, object, presentation, and methodology, and also examines how interpretability applies to Multimodal Large Language Models (AI systems that process and combine text, images, and other data types).
Researchers developed PING (Positive-Incentive Noise Generator), a new method that adds carefully designed noise to protect private data in decentralized learning (where multiple computers train AI models together without sending raw data to a central server) while keeping the learning process efficient. The method uses network connections and lightweight encryption to create correlated noise (noise patterns that work together), and builds on this to create PP-DPIN, an algorithm that combines differential privacy (a mathematical technique for protecting individual data points) and information theory to ensure strong privacy guarantees for at least half the computers involved.
AI systems receive instructions from multiple sources (system policies, developers, users, and online data), and models must learn to prioritize the most trustworthy ones to stay safe. When models treat untrusted instructions as authoritative, they can be tricked into revealing private information, following harmful requests, or falling victim to prompt injection (hidden malicious instructions hidden in input data). OpenAI's solution uses a clear instruction hierarchy (System > developer > user > tool) and trains models with IH-Challenge, a reinforcement learning dataset designed to teach models to follow high-priority instructions even when lower-priority ones conflict with them.
Fix: OpenAI's models are trained on a clear instruction hierarchy where System instructions have highest priority, followed by developer instructions, then user instructions, then tool outputs. The company also created IH-Challenge, a reinforcement learning training dataset that generates conversations with conflicting instructions where high-priority instructions are kept simple and objectively gradable, ensuring models learn to prioritize correctly without resorting to useless shortcuts like over-refusing benign requests.
OpenAI BlogAGFPS is a new attack method that steals system prompts (the hidden instructions that control how an LLM behaves) from deployed AI applications by using evolutionary optimization (a technique that mimics natural selection to find solutions) instead of gradient-based methods. The researchers demonstrated that their approach successfully extracted prompts 95.2% of the time and worked better than previous methods, highlighting serious security weaknesses in how LLMs are currently deployed.
Researchers demonstrated a practical differential fault attack (an exploit that deliberately introduces errors into a system to extract secrets) against GEA-1 and GEA-2, the stream ciphers (algorithms that encrypt data bit-by-bit) used to protect GPRS (General Packet Radio Service, a mobile data standard) communications between phones and base stations. By identifying the exact location where faults occur in the cipher, attackers can recover the 64-bit secret keys in about 16 minutes on a standard laptop. Many current phones still support these outdated ciphers, making them vulnerable.
Game-theoretic DoS attacks (GDoS, attacks that exploit miners' financial incentives) can damage proof-of-work blockchains (like Bitcoin, which uses computational puzzles to secure transactions) even when attackers control less than 20% of the network's computing power. Rather than changing the blockchain protocol itself, researchers propose a cooperative defense where miners temporarily move their computing resources to larger mining pools during attacks to maintain their earnings and discourage attackers.
Fix: The source proposes a 'cooperative hash-power hopping mechanism in which miners temporarily reallocate hash power to larger pools when under attack to preserve expected payoffs and suppress attacker incentives.' Simulations show this strategy 'reduces attacker revenue gains by more than 20% and prevents throughput degradation across the entire attack range.' However, this is a theoretical proposal presented in a research paper, not an implemented or deployed mitigation in existing systems.
IEEE Xplore (Security & AI Journals)Phishing attacks are a form of social engineering (tricking people into revealing secrets by pretending to be trustworthy) that trick users into visiting fake websites that look like real ones to steal sensitive information. Researchers created a new dataset with 31 attributes (measurable characteristics) derived from URLs and similarity features, then tested multiple machine learning algorithms (computer programs that learn patterns from data) on it to detect these attacks. The Logistic Regression method achieved 96.40% accuracy at detecting phishing, showing that this approach works well for protecting local systems in real-world situations.
SeVoAuth is a cloud-based voiceprint authentication system (a security method that recognizes users by their unique voice characteristics) designed to protect user privacy while defending against replay attacks (replaying a recorded voice), spoofing (faking a voice), and adversarial attacks (manipulating input to fool the system). The system stores a synthesized version of a user's voice in the cloud and uses hash functions (mathematical functions that transform data into fixed-size codes) to continuously change the verification targets during each login, making it difficult for attackers to reuse old voice recordings or tricks.