Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
This research proposes a Fairly Proportional Noise Mechanism (FPNM) to address a problem in differential privacy (DP, a technique that adds random noise to data to protect individual privacy while allowing statistical analysis). Traditional DP methods add noise uniformly without considering fairness, which can unfairly affect different groups of people differently, especially in decision-making and learning tasks. The new FPNM approach adjusts noise based on both its direction and size relative to the actual data values, reducing unfairness by about 17-19% in experiments while maintaining privacy protections.
WiFi-based sensing systems that use deep learning (AI models trained on large amounts of data) are vulnerable to adversarial perturbation attacks, where attackers subtly manipulate wireless signals to fool the system into making wrong predictions. Researchers developed WiIntruder, a new attack method that can work across different applications and evade detection, reducing the accuracy of WiFi sensing services by an average of 72.9%, highlighting a significant security gap in these systems.
This research proposes a new system that combines blockchain (a decentralized ledger that records transactions) with zero-knowledge proofs (cryptographic methods that prove something is true without revealing the underlying data) to make AI model inference more trustworthy and private. The system verifies both where the input data comes from and where the AI model weights (the learned parameters that control how an AI makes decisions) come from, while keeping user information confidential. The authors demonstrate their approach with a privacy-preserving transaction system that can detect suspicious activity without exposing private data.
OWASP has released a Top 10 list of security risks specifically for agentic AI applications, which are autonomous AI systems that can use tools and take actions on their own. This framework was built from real incidents and industry experience to help organizations secure these advanced AI systems as they become more common.
The OWASP GenAI Security Project (an open-source community focused on AI safety) has released a list of the top 10 security risks for agentic AI (AI systems that can take actions independently). This guidance was created with input from over 100 industry experts and is meant to help organizations understand and address threats to AI systems.
One-Time Passwords (OTPs, temporary codes used in two-factor authentication to verify your identity) like HOTP and TOTP have vulnerabilities that let attackers bypass security if they steal the secret key stored on a device or server. This paper proposes HP-OTP, a new OTP scheme that combines your password with the device's unique identifier to make it harder for attackers to forge codes even if they compromise either the device or server.
Deep learning attacks have successfully cracked CAPTCHAs (automated tests that distinguish humans from bots) that use large character sets, especially those with alphabets from languages like Chinese. This paper proposes ACG (Adversarial Large Character Set CAPTCHA Generation), a framework that makes CAPTCHAs harder to attack by adding adversarial perturbations (intentional distortions that confuse AI recognition systems) through two modules: one that prevents character recognition and another that adds global visual noise, reducing attack success rates from 51.52% to 2.56%.
Fix: The paper proposes ACG (Adversarial Large Character Set CAPTCHA Generation) as a defense framework. According to the source, ACG uses 'a Fine-grained Generation Module, combining three novel strategies to prevent attackers from recognizing characters, and an Ensemble Generation Module to generate global perturbations in CAPTCHAs' to strengthen defense against recognition attacks and improve robustness against diverse detection architectures.
IEEE Xplore (Security & AI Journals)Federated learning (a system where decentralized devices train a shared AI model together while keeping their data local) is vulnerable to poisoning attacks, where malicious participants inject false data to corrupt the final model. This paper proposes AdaptiveShield, a defense system that uses dynamic detection strategies to identify attackers, automatically adjusts its sensitivity thresholds to handle different attack types, reduces damage from missed attackers by adjusting hyperparameters (settings that control how the model learns), and hides user identities through a shuffling mechanism to protect privacy.
Fix: AdaptiveShield employs: (1) dynamic detection strategies that assess maliciousness and dynamically adjust detection thresholds to adapt to various attack scenarios; (2) dynamic hyperparameter adjustment to minimize negative impact from missed attackers and enhance robustness; and (3) a hierarchical shuffle mechanism to dissociate user identities from their uploaded local models, providing privacy protection.
IEEE Xplore (Security & AI Journals)This paper presents Test-Time Correction (TTC), a system that helps autonomous vehicles fix detection errors while driving, rather than waiting for retraining. TTC uses an Online Adapter module with visual prompts (image-based descriptions of objects derived from feedback like mismatches or user clicks) to continuously correct mistakes in real-time, allowing vehicles to adapt to new situations and improve safety without stopping to retrain the system.
Researchers developed a new method for backdoor attacks (techniques that manipulate AI systems to behave in specific ways when exposed to hidden trigger patterns) that works better in real-world physical scenarios. The method, called VSSC triggers (Visible, Semantic, Sample-specific, and Compatible), uses large language models, generative models, and vision-language models in an automated pipeline to create stealthy triggers that can survive visual distortions and be deployed using real objects, making physical backdoor attacks more practical and systematic than manual methods.
This research paper addresses generalized out-of-distribution detection (OOD detection, where an AI system identifies inputs that are very different from its training data), which is important for AI systems used in safety-critical applications. Rather than focusing on designing better scoring functions, the authors propose a new decision rule called the generalized Benjamini Hochberg procedure that uses hypothesis testing (a statistical method for making decisions about data) to determine whether an input is out-of-distribution, and they prove this method controls false positive rates better than traditional threshold-based approaches.
This research proposes a new framework for side-channel analysis (SCA, a type of attack that exploits physical information like power consumption or timing to break cryptography) by combining multiple different leakage models (ways of measuring how a cryptographic device leaks secrets) using ensemble learning (combining many weaker models into one stronger one). The framework improves how well attackers can recover secret keys by using deep learning with complementary information from different measurement approaches, and the authors prove mathematically that their ensemble model gets closer to the true secret distribution.
Remote attestation (RA, the process of verifying that software running on a trusted computer processor is genuine and hasn't been tampered with) traditionally relies on a single central authority to verify trust, which creates security vulnerabilities. This paper introduces Janus, a new RA system that spreads trust across multiple parties using physical hardware features (PUF, or physically unclonable function, unique identifiers built into computer chips) and smart contracts (automated programs running on blockchain networks) to make the verification process more secure, flexible, and resistant to attacks.
Graph Neural Networks (GNNs, machine learning models that work with interconnected data) perform poorly at detecting anomalies in graphs because of high Class Homophily Variance (CHV), meaning some node types cluster together while others scatter. The researchers propose HEAug, a new GNN model that creates additional connections between nodes that are similar in features but not originally linked, and adjusts its training process to avoid generating unwanted connections.
Fix: The proposed mitigation is the HEAug (Homophily Edge Augment Graph Neural Network) model itself. According to the source, it works by: (1) sampling new homophily adjacency matrices (connection patterns) from scratch using self-attention mechanisms, (2) leveraging nodes that are relevant in feature space but not directly connected in the original graph, and (3) modifying the loss function to punish the generation of unnecessary heterophilic edges by the model.
IEEE Xplore (Security & AI Journals)Smart grids (power distribution systems that communicate usage data electronically) currently use classical public-key cryptosystems (encryption methods based on mathematical problems that are hard to solve) to protect power consumption information, but quantum computing threatens to break these systems. This paper proposes QC-EAM, a new security model using quantum encryption and quantum Fourier transformation (a quantum algorithm for processing data) to protect smart grid communications, tested on IBM's quantum computing platform.
Researchers discovered a serious weakness in tools designed to detect third-party libraries (external code that apps use) in Android applications. They created LibPass, an attack method that generates tricked versions of apps that can fool these detection tools into missing dangerous or non-compliant libraries, with success rates reaching up to 99%. The study reveals that current detection tools are not robust enough to withstand intentional attacks, which puts users at risk since unsafe libraries could hide inside apps.
AI-generated image forgeries created by tools like GANs (generative adversarial networks, AI models that create fake images) are hard to detect reliably, especially when facing new types of fakes or noisy images. Researchers found that forgery detectors fail because of frequency bias (a tendency to focus on certain patterns in image data while ignoring others), and they developed a frequency alignment method that can either attack these detectors or strengthen them by removing differences between real and fake images in how they look at the frequency level.
Fix: The source proposes a two-step frequency alignment method to remove the frequency discrepancy between real and fake images. According to the text, this method 'can serve as a strong black-box attack against forgery detectors in the anti-forensic context or, conversely, as a universal defense to improve detector reliability in the forensic context.' The authors developed corresponding attack and defense implementations and demonstrated their effectiveness across twelve detectors, eight forgery models, and five evaluation metrics.
IEEE Xplore (Security & AI Journals)Current password strength meters in IoT systems (internet-connected devices) incorrectly rate passwords as secure when they contain certain number patterns, causing users to create passwords that are actually weak. Researchers discovered that numbers in passwords follow predictable semantic patterns (like common sequences or meaningful digit combinations), which attackers can exploit using improved PCFG attacks (a method that guesses passwords by learning common patterns from leaked databases). The study proposes updating password strength meters to account for these digit patterns when evaluating password security.
Fix: The source proposes "a feasible scheme to improve the password strength meter for IoT systems based on the high-frequency semantic characteristics of digit segments" but does not provide specific implementation details, code, or concrete steps in the text provided.
IEEE Xplore (Security & AI Journals)N/A -- This content is a website navigation menu and product listing for GitHub's development platform features, not a technical article about an AI/LLM issue, vulnerability, or problem.
This research proposes a new method for protecting data privacy in deep learning (training AI models on sensitive data) by adding Gaussian noise (random values from a bell-curve distribution) to ResNets (a type of neural network with skip connections). The method aims to provide differential privacy (a mathematical guarantee that an individual's data cannot be easily identified from the model's results) while maintaining better accuracy and speed than existing privacy-protection techniques like DPSGD (differentially private stochastic gradient descent, a slower privacy-focused training method).