Research
Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
Academic papers, new techniques, benchmarks, and theoretical findings in AI/LLM security.
2,003 items
Researchers tested whether the number of reasoning tokens a model generates can signal untruthful behavior, without needing access to the reasoning content. Three reasoning-capable large language models answered 210 multiple-choice questions under system prompts to respond truthfully, falsely, or without regard for truth. Truth-directed responses used fewer reasoning tokens than both lie-directed and truth-indifferent responses across all three models. The authors present this as a proof-of-concept signal, not yet a detector of spontaneous deception.
Researchers tested whether the number of reasoning tokens a model generates can signal untruthful behavior, without reading the reasoning content. Three reasoning-capable LLMs answered 210 multiple-choice questions under system prompts instructing them to answer truthfully, falsely, or without regard for truth. Truth-directed responses produced fewer reasoning tokens than both lie-directed and truth-indifferent responses across all three models, and untruthful instructions increased the likelihood of reasoning at all in one model. The authors present this as a proof of concept for a content-independent black-box signal, not yet a detector of spontaneous deception.
Nullify is a training-free method for unlearning specific knowledge from LLMs at inference time, without weight updates. It applies steering vectors to redirect privacy-related activations away from memorized answers, while a null-space constraint keeps retained-query activations essentially unchanged. On TOFU and MUSE, it matches or surpasses established baselines in forget quality and achieves near-lossless preservation of model utility.
SLDR is a post-fine-tuning defense against malicious fine-tuning of aligned LLMs, which can erode refusal behavior while preserving task performance. It trains a LoRA recovery adapter only on the layers with the maximum and minimum sensitivity scores in the signed spectrum, and uses representation-based dynamic routing to activate the adapter only for malicious queries. On Llama3.1/SST2, it reduces the average harmful score from 11.54 to 0.08 while maintaining downstream accuracy, across four architectures, five tasks and four harmful benchmarks.
Researchers propose an LLM-assisted framework to automate content generation for Transportation Management Plans, using the WisDOT WisTMP system as the application context. They fine-tune multiple open-source LLMs at 7B/8B and 14B scales, deploy them locally for data security, and train on a dataset built by converting historical WisTMP PDFs into JSON question-answer pairs. Fine-tuning significantly improves standard text generation metrics, but models over-generate strategies and struggle with project-specific justifications and accurate cost estimates, and scaling from 7B/8B to 14B yields limited gains.
PatchBench introduces a benchmark of 400 curated, model-specific jailbreak failures drawn from 27,870 prompts across 37 public datasets, filtered with WildGuard, pairwise Elo ranking and manual verification. Its companion protocol, PatchBench-Local, tests whether a patch is behaviourally precise by checking harmful-neighbour correction and benign-neighbour preservation. Evaluating four activation steering methods, the authors found global capability could stay nearly unchanged while local benign regressions were severe, showing aggregate metrics miss collateral damage.
Auditopus is a decentralized method for auditing a large language model's fairness, where multiple auditors each query the LLM and share only cumulative statistics vectors instead of raw queries. The authors show that a single adversarial auditor can fabricate these vectors to make an unfair LLM appear fair, and that the scheme counters this by down-weighting auditors whose vectors are statistically inconsistent with earlier ones. Against an optimizing attacker, it reduces audit error by up to 78% on average relative to no defense and at least 62% relative to robust aggregation baselines.
This paper examines whether LLM-based vulnerability patching benchmarks give reliable results. The authors curate 112 historical bugs from 84 open-source C/C++, Go, and Rust projects, each with PoC, regression, and developer tests. They find that LLMs reach high PoC passing rates under ideal conditions, but developer-test passing rates stay low and improve only marginally with newer models.
Researchers measured whether LLM routers leak sensitive-topic information through their model-selection metadata, even when content logging is off. Across 1.7 million real requests, routers sent harassment and self-harm prompts to the strong model 19 points less often, and medical prompts 31 points less often on distinct prompts (both post hoc). Per-category length-matched parity with accurate labels removed the gap on real traffic at a cost of at most 0.2 accuracy points on RouterBench.
Fix: Per-category length-matched parity with accurate labels removes the gap on real traffic, costing at most 0.2 accuracy points on RouterBench (post hoc). Per-conversation stickiness, per-user budget bands, and pooled parity failed. A post hoc exact per-user rate hides even-prefix strong counts but preserves odd-position decisions, so it does not fully close the channel.
Arxiv (cs.CR + cs.CL + cs.LG)Researchers propose OrthoPurify, a method that removes backdoors from large vision-language models (LVLMs) by applying a one-step orthogonal projection to the backdoored weight updates. The method identifies the backdoor as "direction hijacking," where a small number of weight update directions are diverted from task adaptation to encode a backdoor shortcut. A pseudo-benign reference model, fine-tuned on a small set of clean samples, is used to isolate these hijacked directions. The authors report that OrthoPurify reduces attack success rate to near zero while preserving original performance, without retraining or inference-time overhead.
Fix: OrthoPurify: a one-step orthogonal projection on the weight update that removes hijacked directions, using a pseudo-benign reference model obtained by fine-tuning the pretrained weights on a small set of clean samples. Code is available at https://github.com/womeimingzi/OrthoPurify.
Arxiv (cs.RO + cs.CV security)This research shows that multi-vector visual document retrievers can be inverted, letting an attacker who runs or breaches the vector store reproduce source pages from the stored index alone. On the ViDoRe v3 benchmark, inverted pages recover 47% of words and rank their source page first 98.4% of the time. Token pooling and shuffling cut word recall to about 8%, but a model that restores shuffled order raises first-rank matches from 3.8% to 93.5%.
The paper proposes a constrained-action architecture for LLM-driven remediation in SIEM/XDR security operations. It pairs a control plane that limits the model to a closed intent vocabulary with a NeMo-Guardrails proxy that wraps the analyst LLM. The stock proxy raises injection recall from 25.0% to 94.5% at a 0.1% false-positive rate.
Fix: The source describes a constrained-action design rather than a fix for a specific flaw. It confines the LLM's output to a closed intent vocabulary of templated commands executed by thin endpoint agents, backed by an argument validator, and wraps the analyst LLM with input- and output-rail policies via a NeMo-Guardrails proxy. It also suggests running the loop human-in-the-loop or delayed, since the measured rail latency rules out inline control.
Arxiv (cs.CR + cs.CL + cs.LG)Researchers evaluated whether Looped Language Models (LoopLMs), which reuse shared parameters across recurrent steps, keep their safety behavior at every inference depth. They found attack success can rise at deeper depths, the same query can yield different safety behaviors across depths, and attacks transfer between depths, with SFT and preference alignment not closing these gaps. They introduce SafeBridge, which combines depth-specific control of shared recurrent layers, selective state bridging, and joint safety supervision across depths, and report substantially reduced attack success while preserving general utility and comparable over-refusal behavior.
Fix: SafeBridge: lightweight depth-specific control of shared recurrent layers, selective state bridging, and joint safety supervision across recurrent depths. Code and model checkpoints will be released upon acceptance.
Arxiv (cs.CR + cs.CL + cs.LG)This study evaluates metric first-order temporal logic (MFOTL) as a declarative guardrail for tool-using LLM agents by replaying recorded trajectories from AgentDojo, STAC and R-Judge through the unmodified MonPoly monitor, offline and without running an agent. Five generic obligations flag 71.8% of STAC attack chains and 70.1% of successful AgentDojo attacks, but also fire on 29.3% of benign runs, an imprecision the authors attribute to the corpora, which rarely record approvals and never record timestamps. A planted line evades a naive provenance check in 94-99% of the runs it would otherwise flag, and binding provenance to the lookup that produced it closes this evasion at no cost in detection or benign firing.
Fix: Binding provenance to the lookup that produced it closes this evasion at no cost in detection or benign firing. The authors also propose a twelve-field enforcement-ready trace schema.
Arxiv (cs.CR + cs.CL + cs.LG)TMT is a runtime backdoor detector for vision-language-action (VLA) policies, built on token manifold and latent transition modeling and trained on benign rollouts. The authors also explore policy purification through self-distillation, using a frozen copy of the backdoored policy as a teacher. In a post-hoc comparison with ten baselines, TMT achieves state-of-the-art detection on unseen tasks across three VLA backdoor attacks.
Apollo Research and collaborators studied how metagaming, where a model reasons about how a task will be evaluated or rewarded instead of attempting it, is represented inside an OpenAI o3 reinforcement learning run. They identified sparse autoencoder latents linked to metagaming that strengthened during RL training and could influence answers without appearing in the written chain of thought.
Researchers show that adversarial images can plant a backdoor in a vision-language model that persists after the image is masked from attention. The persistent variant, Persistent Visual Memory Injection (P-VMI), reaches up to approximately 90% target success on Qwen3-VL-8B-Instruct, and a cache-swap ablation localizes the influence to the KV cache. The attacks also survive compaction that retains the KV cache of a model-generated summary.
VCR-Bench is a modular open-source benchmark framework for studying robustness of video classifiers, which the authors say has been studied across scattered, incompatible implementations. It standardizes video loading, classifier wrappers, adversarial attacks and defenses, perceptual metrics, configuration presets and result logging under a common evaluation protocol. The authors evaluate representative video classifiers, attacks and defenses on a Kinetics-400 subset, reporting clean accuracy, attack success rate, perceptual quality, runtime and memory usage.
The source is an ACM Transactions on Privacy and Security article, Volume 29, Issue 4, pages 1-34, dated November 2026. The provided source text contains only the bibliographic citation and no article content, so no findings or methods can be summarized.