Research
Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
2,033 items
OpenAI released Model Spec Evals, a suite that measures how well models follow the OpenAI Model Spec, along with 596 evaluation prompts and open-source evaluation code. Compliance rates were 72% for GPT-4o, 80% for OpenAI o3, 82% for GPT-5 Instant, 89% for GPT-5 Thinking, 84% for GPT-5.3 Instant, and 87% for GPT-5.4 Thinking. The evaluations cover only text-only interactions, and the prompt collection is small relative to the Spec's scope.
Researchers propose MPOF, a privacy-preserving multi-modal object fusion model for connected autonomous vehicles that fuses LiDAR and camera detections using secret sharing. The model detects malicious behavior by a third-party random number generator through sacrificial verification. The authors report that their protocols reduce computational overhead by five orders of magnitude compared with homomorphic-encryption-based random number generation.
This article addresses composite resource allocation problems with general constraints, common in networked systems such as smart grids and multiagent coordination. It proposes two anti-disturbance proximal neural networks: one for structured disturbances, built on the internal model principle, and an observer-based one for unstructured disturbances. Asymptotic convergence of both is established via Lyapunov stability theory, and numerical simulations validate their effectiveness and robustness.
Researchers propose a differentially private accelerated distributed gradient tracking algorithm for distributed aggregative optimization, where each agent's objective depends on an aggregate of all agents' decisions. The method perturbs exchanged information with independent Laplace noise, adds a noise deduction mechanism to keep estimates accurate, and combines dynamic average consensus with heavy-ball momentum. Under strong convexity and Lipschitz-continuous gradients, the authors prove linear convergence in mean-square error, derive explicit suboptimality bounds, and establish that the algorithm satisfies epsilon-DP; numerical simulations are provided.
This article presents a systematic review and meta-analysis of research on GAN-based adversarial attacks against intrusion detection systems (IDSs). The authors empirically test binary and multiclass IDS models under GAN-based attacks, examining the influence of feature selection. They assess IDS performance and sample realism using Fréchet inception distance and Hellinger distance, and identify the most severe GAN architecture and the one that generates the most realistic adversarial samples.
TrustTiny-HAR is a pipeline for on-device human activity recognition on microcontrollers that pairs an int8 time-series backbone with a distilled trust head and a streaming conformal layer. Evaluated with leave-one-subject-out protocols on UCI HAR, PAMAP2 and WISDM, it matches int8 baselines on in-distribution macro-F1 (93.8 on UCI HAR) while improving calibration (ECE 2.4%) and out-of-distribution robustness (AUROC 0.96 for unseen classes). On nRF52840 and STM32L4 it adds under 10% energy overhead versus a TinyCNN baseline.
Researchers propose TSFA, a two-stage feature alignment method for unsupervised open-set domain adaptation in time-series classification. The method combines a time-frequency feature extractor with global alignment via similarity distribution entropy and local alignment via pseudolabel-guided self-supervised learning. Experiments on three real-world datasets show it outperforms state-of-the-art approaches.
This paper presents a reinforcement learning framework that guarantees robustness under bounded uncertainties in dynamics modeling and state estimates. The uncertainties and system dynamics are described with ReLU neural networks, which let the method find the states that most violate the robustness conditions and prioritize them during training. The authors evaluate the approach in simulation, including 3-D quadrotor reference trajectory tracking.
This research paper analyzes how backdoor attacks work against semi-supervised learning (SSL), finding that attackers exploit pseudo-labeling to build stronger trigger-target correlations and that triggers can appear in low, medium, or high frequency bands. The authors propose Backdoor Invalidator (BI), a defense combining complementary learning, trigger mix-up, and dual domain filtering. Evaluation against state-of-the-art attacks shows BI significantly reduces average attack success rate while keeping comparable clean-data accuracy.
Fix: BI, a defense framework integrating complementary learning, trigger mix-up, and dual domain filtering, which the authors describe as a plug-in component. Code is available at https://github.com/wxr99/Backdoor_Invalidator4SSL
IEEE Xplore (Security & AI Journals)This paper proposes a Propose-Rectify framework for image manipulation localization that pairs a forensic-adapted LLaVA model with a Forensics Rectification Module and an Enhanced Segmentation Module that adds forensic cues to SAM's image embeddings. The authors report state-of-the-art performance across diverse datasets, with strong robustness and generalization, though the source text gives no numeric results.
Researchers propose a redundant fully connected layer that can be plugged into existing image classification model backbones to improve adversarial robustness. The layer is trained with a loss function that uses cosine similarity to maximize the difference and diversity among multiple fully connected parts. Tested against 12 representative white-box and black-box attacks on two popular datasets, the approach reportedly gives significant robustness with negligible extra training overhead and hardly degrades clean-sample accuracy.
Fix: The proposed mitigation is the redundant fully connected layer, combined with existing model backbones in a pluggable manner, trained with the cosine-similarity-based loss function described in the source.
IEEE Xplore (Security & AI Journals)Researchers propose MOSA-S2, a multiobjective simulated annealing method that generates text rubbish examples by substituting input words with meaningless stopwords. The method uses importance-based composite perturbation and a grammatically constrained variant, and it outperforms prior approaches across six text datasets and seven neural models. The authors report that models often keep the same prediction, sometimes with higher confidence, on nonsensical text.
FedDOT is a federated learning framework that defends against targeted attacks, including attacks by adversaries who make up more than half of the participants. It combines two detection algorithms, MST-AD (maximum spanning tree-based attacker detection) and density-AD (densest graph-based attacker detection), which use correlation between weight updates and graph theory. Experiments on image classification datasets in non-IID settings report an attack success rate below 10% for single-label flipping, multilabel flipping and backdoor attacks, with an accuracy drop under 2%.
TrustAI is an expert-based methodology for evaluating and certifying the trustworthiness of AI systems. It decomposes high-level trust pillars into low-level criteria that AI experts answer, aggregates those judgments through a hierarchical model into a quantitative level of trust (LoT), and issues a certificate when results are satisfactory. The authors illustrate it on two actual AI systems using a model based on a recently issued regulatory framework.
The authors introduce SRAP, a Self-Reversible Adversarial Patch technique for generating reversible adversarial examples to protect images from malicious deep model identification and analysis. Combining small, non-overlapping adversarial patches with prediction error expansion data hiding, the method reports transferability of up to 90% or higher between models and an 88% attack success rate on commercial APIs.
The authors analyze CLIP's multi-stage image embedding space and find that mid-level stages encode forensic and generator artifact features, while high-level stages encode semantic artifact features. They propose CLIP-ADA, which combines dual-level augmentation to reduce reliance on artifact features with forensic distribution adaptation that detects deviations from CLIP-encoded real embeddings. The method outperforms state-of-the-art detectors by over 6% average accuracy on unseen data from more than 10 generators.
Researchers Bruce W. Lee, Yueh-Han Chen and Tomek Korbak train agents to call a report_scheming() tool whenever they covertly misbehave, a method they call self-incrimination. For GPT-4.1, undetected successful attacks fall from 56% to 6%, outperforming matched-capability blackbox monitors and alignment baselines across 15 out-of-distribution environments. The approach also preserves general capabilities and generalizes from instructed to uninstructed misbehavior.