Research

This source describes how an assessor could investigate whether unreleased near-frontier AI models might pursue serious misaligned actions, such as unsanctioned cyberattacks. The proposed method reviews internal transcripts from training, evaluation, and internal use, and examines whether evaluations and counterfactual simulations understate the worst behaviors agents might take. The assessment would rely mainly on access to comprehensive internal transcripts and sampling access to internal model checkpoints.

This article describes how an assessor could investigate whether AI capabilities progress has accelerated because of AI-driven automation of AI R&D. The assessment would examine capability benchmarks, intermediate indicators such as pretraining loss, and researcher perceptions, and would analyze unreleased models along with the growth of labor, compute, data and algorithmic inputs. It would also assess bottlenecks and whether the methods would reliably detect a dramatic acceleration.

METR proposes that independent assessors give the public evidence on whether frontier AI developers could lose control of their internal AI systems. The post argues that the public lacks visibility into internal development, and it describes three evidence gaps, covering alignment, control, and AI R&D acceleration. It proposes that developers work with METR on risk assessment exercises to share this evidence.

METR, an evaluation organization, describes how it works with AI companies on projects that inform public understanding of AI loss-of-control risk. Its stated goal is to give the public accurate, timely, independent evidence so the public, policymakers and developers can make informed decisions. The post, published Oct 14, 2026, lists transparency commitments covering scope, access, redactions, constraints, conflicts of interest and confidentiality.

Researchers show that LLMs can be morally reframed to bypass safety guardrails. Their multi-agent framework, grounded in Moral Foundations Theory (MFT), diagnoses which moral foundation a refused harmful prompt violates, finds a foundation under which the request seems virtuous, and rewrites the prompt with generated precedents. Across five open-source and proprietary models and prompts from three jailbreak datasets, models proved highly susceptible, with success rising under repeated attempts.

Researchers present Impossible Mission Finder (IMF), a multi-agent system that combines static analysis, dynamic execution and log analysis to automatically identify, test and validate logic flaws in autopilot software. Applied to PX4 and Ardupilot, two widely used open-source autopilot systems, it found 105 previously unknown logic flaws that can cause unsafe behaviors such as mission interruption failures, invalid actions and entry into restricted fly zones.

GraphRAG-LTM is a Layered Threat Modeling framework that tests three nested levels of attacker access against graph retrieval-augmented generation (GraphRAG). Under black-box access, ranking feedback guides query edits that demote relevant seed chunks, and white-box retriever gradients make that attack stronger. A graph-aware white-box variant also deletes bridge triplets from the query-specific evidence graph, which further impairs supporting-fact retrieval.

This research examines whether quantized GGUF models on Hugging Face can be independently re-derived from their published information. Across the 200 most-downloaded GGUF repositories (198 in scope), none record the build environment, so the Independent Verifiability Rate under a recorded-inputs protocol is 0/198. On four repositories that disclose their inputs, two omitted coordinates, the floating-point-contraction setting and the per-tensor quantization recipe, reproduce 7 of 8 target-quant files bit-for-bit on x86.

MMA-SafetyBench is a benchmark for testing whether multimodal agents can be steered off their authorized task by deceptive visual interface elements, such as forged system alerts. The benchmark contains 1,083 adversarial trajectories across five domains, and across nine evaluated models the Attack Success Rate reaches 87.96% while the Semantic Compromise Rate reaches 98.09%. Attacks persisted under neutral prompts, and a safety-priority prompt lowered the Attack Success Rate without eliminating the failure mode.

Runtime policy verifiers usually discard their enforcement decisions, but this paper asks whether those verdicts can train the agent instead. The authors propose Policy-Verified Self-Training (PVST), where a deterministic gate compiled from a written policy scores the agent's own trajectories, reinforcing compliant ones and using violations as negatives for preference optimization. On held-out AgentDojo tasks under prompt injection, training Qwen3-8B cut violating trajectories from 37.2% to 10.3% and successful attacks from 12.4% to 0.5%, though utility fell on travel tasks.

Researchers propose Topology-Aware Dynamic Co-evolution (TADC), a framework that defends Graph Neural Networks against adversarial perturbations of graph structure by jointly optimizing graph topology and model parameters through validation-guided meta-gradients. The method adds a topology-aware gradient calibration mechanism, localizes vulnerable nodes, and separates reliable edges from potentially adversarial ones before training an attention-based DeepGCN. The authors report that TADC consistently outperforms state-of-the-art methods across attack settings on multiple benchmark datasets, with the largest gains under strong perturbations and high perturbation rates.

This research examines how jailbreak evaluation depends on the judge LLM, showing that swapping only the judge can change reported attack success rate by up to 82 percentage points. The authors introduce the semantic reconstruction game, a training-free evaluator that reconstructs the request a response answers and compares it against decoy requests, with abstention when neither matches. Across three off-the-shelf LLM judges it achieves the lowest mean error against human labels among the methods evaluated and reduces variation in reported ASR, prioritizing avoidance of false positives.

AgentCIScope is a platform and benchmark that tests whether LLM agents preserve contextual integrity when they read and write sensitive data across application environments through tool calls. The authors evaluated 12 agents on 936 executable tasks and found single-run leakage rates from 32.5% to 94.4%, rising by 4 to 19 percentage points when each task is repeated five times. Leakage varies by attribute, with personal preferences at 84.7% and credit-card information at 17.1%.

The TRACER paper shows that indirect prompt injection succeeds far more often when the malicious text sits in retrieved context (53%) than when it arrives as a follow-up message (18%). The authors propose TRACER, a channel-aware adversarial training method that combines an answer-anchored, evidence-gated reward with ATRC. On GSM8K with Qwen2.5-3B-Instruct, it cuts RAG-framed attack success from 65.5% to 6.1% while keeping 78-82% clean accuracy.

DecisionRobust is a benchmark that evaluates the robustness of JEV-style decision models on text and vision tasks, using semantics-preserving text perturbations, image perturbations, and prompt injection in both modalities. The authors test 30 text and 17 vision model configurations, including held-out models excluded from candidate selection. They find that attacks expose vulnerabilities beyond the construction models, that resistance to input perturbation does not imply resistance to prompt injection, and that label-consistency checks substantially reduce measured attack success.

CyberArena is a benchmark that pits two AI agents against each other in live, symmetric attack-and-defense matches inside isolated sandboxes, with services that change over time. Across 466 matches involving 12 frontier LLM agents and 40 challenges, it separates model capabilities that single-agent static benchmarks fail to distinguish. The authors report that defenders often patch one flaw while missing the next, and that stronger agents plant backdoors for later rounds.

The authors present AIRRI, a framework that makes educational assignment images machine-resistant through character and word spacing, opacity variation, stripe overlays and noise, plus decision-based adversarial attacks (RayS, SurFree, ADBA). Tested against Tesseract, EasyOCR and GOT-OCR2, it raised character error rates for EasyOCR from 6.9% to 55.9%, Tesseract from 2.3% to 100.0%, and GOT-OCR2 from 30.7% to 48.1%.

Researchers studied contextual entrainment, where language models reuse irrelevant tokens from earlier prompt context in later answers. Using factual-distractor relation-completion prompts across 19 lexical-semantic subtypes, they identified attention heads whose ablation reduces entrainment and tested how those head sets transfer across exact relations, semantic families, and mixed distributions. They found that same-family transfer exceeds different-family transfer and that subtype-specific searches repeatedly rediscover many globally shared heads, which can reduce harmful copying while preserving ordinary relation completion.

DPOison-VL is a clean preference poisoning attack against vision-language models trained with Direct Preference Optimization (VLM-DPO). It applies imperceptible perturbations to a small share of training images, leaving prompts, responses and annotations intact, and reaches up to 98% ASR with 2% poisoned data. The authors report cross-model transferability while clean accuracy and general utility are largely preserved.