Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
This research paper studies whether deepfake detection systems (AI tools that identify fake videos made to look real) are fair across different groups of people based on ethnicity, gender, and age. The study found that these detection systems often perform differently depending on the person's background, meaning they work better for some groups than others. The paper highlights that bias in deepfake detection is an important fairness problem that needs attention.
This academic paper discusses using explainable AI (AI systems that can show their reasoning for decisions) to detect financial fraud as it happens in real time. The research focuses on making fraud detection systems that adapt to new fraud patterns while also being transparent about why they flag transactions as suspicious.
This research paper proposes a new method for detecting deepfakes (AI-generated fake videos or images) by using neural networks (computer systems loosely modeled on how brains learn) trained with both current and quantum computing approaches. The dual approach aims to make deepfake detection more reliable and harder for attackers to bypass.
This research article proposes a hybrid machine learning approach to improve trust management and security in mobile crowdsourcing (a system where mobile users contribute data or complete tasks for a distributed project). The study combines multiple machine learning techniques to identify trustworthy participants and protect against malicious actors in crowdsourcing environments.
AGentVLM is a framework that uses small language models (AI systems trained on text) to automatically convert written organizational rules into access control policies (rules defining who can access what resources). The system avoids using large third-party AI services, keeping data private, and can handle complex requirements like purposes and conditions while verifying that generated policies are accurate before they're put into use.
Federated learning (a system where multiple participants train a shared AI model without sharing their raw data) is vulnerable to attacks from malicious clients who send harmful model updates. This paper proposes AMF-CFL, a defense method that uses multi-k means clustering (a technique for grouping similar data points) and z-score statistical analysis (a way to identify unusual values) to filter out malicious updates and protect the global model, even when clients have non-i.i.d. data distributions (when each participant's data differs significantly in type and quantity).
Fix: AMF-CFL reduces the influence of malicious updates through a two-step filtering strategy: it first applies multi-k means clustering to identify anomalous update patterns, followed by z-score-based statistical analysis to refine the selection of benign updates.
Elsevier Security JournalsThis research paper presents XAIDroid, a framework that uses graph neural networks (GNNs, machine learning models that analyze relationships between connected pieces of data) and graph attention mechanisms to automatically identify and locate malicious code within Android apps. The system represents app code as API call graphs (visual maps of how different functions communicate) and assigns importance scores to pinpoint which specific code sections are malicious, achieving high accuracy rates of 97.27% recall at the class level.
Fed-Adapt is a federated learning framework (a system where multiple computers learn together while keeping their data private) designed to defend networks against DDoS attacks (floods of traffic meant to overwhelm servers) and database flooding attacks (requests that exhaust database resources). The framework addresses the challenge of detecting and responding to these sophisticated attacks in real-time while protecting data privacy across distributed networks, which existing federated learning approaches struggle to do effectively.
This is a review article examining how Large Language Models (LLMs, AI systems trained on vast amounts of text to understand and generate language) are being used in cybersecurity to analyze malware (harmful software designed to damage systems). The article surveys recent research on using LLMs for malware detection, understanding malicious code structure, reverse engineering (the process of analyzing compiled software to understand how it works), and identifying patterns of malicious behavior.
Federated learning (a system where multiple computers train AI models together without sharing their raw data) faces two major security problems: attackers can steal information from the local models that clients upload, and malicious clients can sabotage the training by sending bad models. This paper proposes VFEFL, a new federated learning scheme that uses verifiable functional encryption (a type of encryption that lets you check if calculations on encrypted data are correct without decrypting it) to protect client data privacy while detecting and defending against attacks from dishonest participants.
Fix: The paper proposes VFEFL (a privacy-preserving federated learning scheme based on verifiable functional encryption) as the solution. According to the source, VFEFL 'employ[s] a verifiable functional encryption scheme to encrypt local models in the federated learning, ensuring data privacy and correctness during encryption and decryption' and 'enables verifiable client-side aggregated weights and can be integrated into standard federated learning architectures to enhance trust.' The source states that 'experimental results demonstrate that VFEFL effectively defends against such attacks while preserving model privacy' under both targeted and untargeted poisoning attacks.
Elsevier Security JournalsThis research paper proposes FIPAPNet, a machine learning system designed to classify malware when only a few samples are available, which is important because new malware variants often appear with limited examples. The system uses few-shot learning (a technique where AI learns from minimal training data) combined with dynamic features like system call sequences to achieve 93% accuracy in early-stage malware detection. This approach helps security defenders respond quickly to zero-day attacks (new, previously unknown malware) without needing hundreds of samples to retrain their detection models.
Vuln2Action is an LLM-based framework designed to help security testers reproduce vulnerabilities and map exploits more systematically. The paper addresses a key challenge in penetration testing (controlled simulations of cyberattacks to find security weaknesses): vulnerability reproduction is time-consuming and relies heavily on manual expertise, yet publicly available exploits exist for less than 1% of known vulnerabilities. While LLMs show promise for analyzing large amounts of threat data, the authors found that current models often refuse to provide exploit-related guidance due to built-in safety restrictions.
This paper discusses the growing challenge of malware (malicious software designed to exploit computer system vulnerabilities) detection, noting that over 450,000 new malware samples are detected daily as of 2024. Traditional detection methods like signature-based detection (matching known byte patterns against a database) and behavior-based detection (running malware in isolated test environments to observe its actions) have limitations: signature-based methods fail against new or disguised malware, while behavior-based methods are computationally expensive and can be evaded by malware that detects virtual environments. The paper proposes using machine learning and deep learning approaches trained on features from both static and dynamic analysis to better classify files as malicious or benign.
This paper discusses differential privacy (DP, a mathematical method that adds noise to data to protect individual privacy while keeping data useful), which is stronger than traditional anonymization techniques like generalization and suppression. The authors address a key challenge: existing DP methods struggle with high-dimensional data (datasets with many features) and treat all data features equally, even though real-world data has varying privacy needs, such as medical records where disease diagnoses need more protection than age.
This item is a biography of Dr. Kamta Nath Mishra, an academic researcher with over 25 years of experience in computer science. While the title mentions a hybrid machine learning and cryptography model for cloud-IoT (internet of things, networked physical devices) security, the provided content contains only his educational background and career history with no technical details about the actual security research or any vulnerabilities.
This article examines how large language models (AI systems trained on huge amounts of text data) can be used in cybersecurity red teaming (simulated attacks to test defenses) and blue teaming (defensive security work), mapping their abilities to established security frameworks. However, LLMs struggle in difficult, real-world situations because they have limitations like hallucinations (generating false information confidently), poor memory of long conversations, and gaps in logical reasoning.
This research paper explores vulnerabilities in Pedestrian Attribute Recognition (PAR), a computer vision task that identifies characteristics of people in images using AI models. The authors developed both adversarial attacks (methods to fool the system with manipulated images) and a defense strategy called semantic offset defense to protect PAR systems, testing their approach on multiple datasets.
Fix: The paper proposes a semantic offset defense strategy to suppress the influence of adversarial attacks on pedestrian attribute recognition systems. Source code is made available at https://github.com/Event-AHU/OpenPAR.
IEEE Xplore (Security & AI Journals)This research paper presents a new method called FoVB (Forgery-aware Audio-Visual Adaptation with Variational Bayes) to detect deepfakes (AI-generated fake videos that manipulate both audio and video). The method works by analyzing the relationship between audio and video to find mismatches, such as when lip movements don't match the sound, which are telltale signs of deepfakes.
Voice authentication on smartphones is vulnerable to spoofing attacks, where attackers replay recorded voice samples through loudspeakers to trick the system. MagLive is a new security method that detects whether a voice is from a real person or a loudspeaker by analyzing magnetic pattern changes (detected by the smartphone's built-in magnetometer) using a machine learning model called TF-CNN-SAF (a type of neural network designed to extract useful patterns from data).
AI systems receive instructions from multiple sources (system policies, developers, users, and online data), and models must learn to prioritize the most trustworthy ones to stay safe. When models treat untrusted instructions as authoritative, they can be tricked into revealing private information, following harmful requests, or falling victim to prompt injection (hidden malicious instructions hidden in input data). OpenAI's solution uses a clear instruction hierarchy (System > developer > user > tool) and trains models with IH-Challenge, a reinforcement learning dataset designed to teach models to follow high-priority instructions even when lower-priority ones conflict with them.
Fix: OpenAI's models are trained on a clear instruction hierarchy where System instructions have highest priority, followed by developer instructions, then user instructions, then tool outputs. The company also created IH-Challenge, a reinforcement learning training dataset that generates conversations with conflicting instructions where high-priority instructions are kept simple and objectively gradable, ensuring models learn to prioritize correctly without resorting to useless shortcuts like over-refusing benign requests.
OpenAI Blog