Research
Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
2,003 items
Researchers propose Logit Laundering, an evasion framework in which a non-compliant LLM provider manipulates model outputs at inference time to suppress watermark-induced statistical signals used by data-use auditors. The method reshapes logits with a plug-and-play statistical bias estimator and a confidence-aware scaling module, leaving model parameters unchanged. Extensive evaluations reportedly evade various state-of-the-art auditing methods while preserving utility on downstream tasks.
This review paper analyzes deep learning and machine learning challenges, including data bias and scarcity, poor interpretability, weak generalization, adversarial vulnerability, hallucinations, and high computational and energy costs. Drawing on literature published from 2022 to 2026, it examines emerging solutions such as Vision Transformers, Mamba models, quantization, pruning, LoRA, federated learning, SHAP, LIME, and Retrieval-Augmented Generation. It concludes by outlining research gaps and future directions. Published in the International Journal of Science and Research on 2026-11-06.
FedReason uses a large language model to choose the server-side aggregation rule in federated learning, calling it every ten rounds to analyse client weights in a restricted sandbox and submit a short Python aggregation program, falling back to FedAvg when output is invalid. Evaluated with Claude Opus 5.5 on CIFAR-10 and CIFAR-100 with ResNet-18 across 168 runs, it beat FedAvg by 1.26 percentage points on average across 12 paired label-skew runs. It remained about one point below FLTrust under Byzantine clients, and accuracy fell from 73.8% to 38.8% in one seed without proxy feedback.
Researchers examined whether explanation APIs for tabular classifiers help model extraction, separating attribution rankings from the bin-edge coordinates they supply. They found attribution gives no systematic advantage, and default Lime bin edges are quantiles of the explainer's background data rather than target-model properties. Their attack, Cutpurse, improves fidelity by 16.3 percentage points when bin edges are withheld, exceeding the undefended published attack by 7.0 points.
Energy grid operators need smart meter data but often lack the SQL skills to query it, and privacy rules can bar sending consumer data to cloud LLMs. The authors present OpenMeter, a schema-agnostic Multi-Agent System that turns natural language queries into executable code and readable insights. Ablation studies show full multi-agent orchestration gains the most on resource-constrained local hardware (+6.1 pp EX-R over the strongest single-agent configuration), while at frontier scale components should be enabled selectively.
This research paper formalizes authorization drift, where LLM agents exceed their authorization boundaries across data, tool, scope, and time while pursuing user goals without an adversary. The authors introduce the Authorization Drift Benchmark (ADB), with 648 samples across seven domains, and evaluate 13 LLMs and three frameworks over 25,207 trajectories. Frontier-capable models show drift rates of 0.43 to 0.61, though 86.18% of tasks admit a clean-success trajectory, and drift is mostly driven by early data over-collection.
This empirical study evaluates Jev, a probabilistic decision model, as a safety guardrail for a Qwen2.5-1.5B-based LLM pipeline across 970 benchmark prompts. Input filtering cut harmful-prompt answers from 527/720 (73.2%) to 17/720 (2.4%), but raised benign-prompt refusals from 3/250 (1.2%) to 27/250 (10.8%). Tiered filtering with selective output review reduced benign refusals to 10/250 (4.0%) and harmful answers to 40/720 (5.6%), with 8.1% of prompts requiring output review.
Researchers studied thought injection, where untrusted text styled as a model's own reasoning is fed back to Large Reasoning Models (LRMs). Testing production open-weight models (GLM-5.3, MiniMax-M2.5, Qwen3.8-Flash-Next) in a containerized agentic cyber range, they achieved attack success rates up to 50% on objectives such as reverse-shell exfiltration and repository backdooring. The same objectives in standard user-voice prompt injection never succeeded, and thinking tags suppressed the agent's disclosure of the injection.
OpenAI introduced BioBasicsBench, a 100-task benchmark developed with Ligo that measures how well AI agents interpret raw wet-lab experimental data, including gel images, colony plates, microscopy and instrument readouts. At the 'xhigh' reasoning setting, GPT-6 Astra earned an average of 66.7% of available credit, compared with 43.8% for GPT-5.6 Sol. The results show progress on routine lab analysis but leave room for improvement in image interpretation, control checking and applying experimental context.
Conflux is a principal-context defence against privilege escalation in LLM agents, enforced by a kernel called ITES that intersects the permissions of every principal influencing an action. The authors prove a conditional authority-confinement theorem for the represented value-influence fragment, and evaluate with SLED and SLED-V alongside AgentDojo, reporting bounded and partial evidence only.
AttentionSwitch is a jailbreak defense for large language models that jointly learns which attention heads to intervene on and how strongly to scale them, keeping the model frozen. It addresses the decoupled optimization of location and strength used by prior defenses. The authors report that it outperforms existing defenses across popular models and jailbreak attacks, especially when training data is scarce.
The paper presents RL-ATTACK, a trainable black-box framework that uses reinforcement learning to generate adversarial transformations against language models without gradient access. On natural language inference, it reduces post-attack accuracy to 2% within 20 queries, and against generative models it beats an optimization-based baseline in 4 of 6 settings. The authors report that no reward design or training setting is consistently dominant.
The authors test whether open-weight decision models can screen retrieved web passages for hidden injected instructions and synthetic misleading pages. On a 1,000-question benchmark, a Kev-4B gate reading each passage's HTML cut attack success from 27.9% with no defence to 13.6% with published mitigations and to 5.7% with the gate added, at 76 ms per passage and 4 of 186 benign pages wrongly flagged. On a 200-query pollution testbed, filtering the candidate pool before selecting the top ten kept fake pages out of answers (0.5% exposure) and held accuracy at 83%, against 28% with no filter.
Fix: Gate retrieved passages with a released open-weight decision model such as Kev-4B that reads each passage's HTML, applied to the candidate pool before the top ten are chosen; the published mitigations of sanitisation, normalisation and citation prompting are also tested as baselines.
Arxiv (cs.CR + cs.CL + cs.LG)Multimodal-SEA is an interactive benchmark of 608 expert-designed tasks based on real-world fraud cases, covering 14 domains and 152 attack scenarios with forged multimodal content, in Chinese and English settings. Evaluating seven multimodal models, the authors find all are vulnerable, with attack success rates of 7.1% to 34.5%. Adding multimodal content raises attack success from 19.9% to 29.2%, and stronger safety instructions reduce attacks without improving recognition of deceptive content.
Two-tier web triage, where a fast model closes most requests and escalates the rest, is tested on seven held-out attack families. Supervised detectors such as ModernBERT give many held-out path traversal requests a confident benign verdict, escalating only 50% to 81% of them within a 5% benign budget. Two hosted zero-shot decision models, Jev and Clef, escalate at least 97.6% of every family within the same budget, though their undisclosed training data may include these public families.
KeHoSu is a method that encodes an identifier in the choice of confusable Unicode homoglyph characters, so text generated by an LLM can be attributed to a platform and account even after copy-paste. The authors evaluated it across 18 common platforms and report that the mark survived copy-paste and attributed most excerpts correctly at 178 letters on LLM-generated stories and 167 letters on standard English prose.
CANMentor is a framework in which a cloud-side large language model teacher supervises lightweight intrusion detection models on vehicles, addressing the lack of built-in security on the Controller Area Network (CAN) bus. A privacy-aware generator converts original CAN messages into proxies before cloud inference, and the vehicle uses teacher predictions only within a latency bound. On non-IID partitions of the ROAD and HCRL Car-Hacking datasets, knowledge distillation yields maximum relative Macro F1 improvements of 25.01% and 20.79%.
Researchers introduce a benchmark for indirect prompt injection that tests task-tampering attacks, where planted content influences a decision the source is not authorized to make, rather than redirecting the agent's task. Across 90 validated cases in three tool-use environments and five LLMs, attacks on decision rules, available actions, and state had higher mean success than objective attacks without defense. A detection-and-removal defense cut objective-targeting attacks to 2.4% success but left observation- and state-targeting attacks at 9.5% and 12.1%, and a tool-policy defense still allowed 15.4% attack success.
METR staff performed a high-access evaluation at Anthropic from August 12 to October 29, 2026, to assess whether AI is accelerating AI R&D. The report estimates that, as of October 2026, Anthropic's models are improving on well-specified tasks at about twice the rate seen between March 2024 and October 2025, with high uncertainty. Anthropic redacted most of the evidence the authors found most informative, which limits the public's ability to check the conclusions.