Research

OpenAI describes how automated red-teaming was used to train GPT-6 Astra and Sol to resist jailbreaks in high-stakes domains such as cyber, biology and violent activity. An attacker model iteratively refines prompts against defender models GPT-5.4 and GPT-5.5, and the resulting synthetic jailbreaks serve as training data. On a fixed cyber jailbreak dataset, GPT-6 Astra answered 91.5% safely versus 59% for GPT-5.6 Sol, measured without production safeguards.

Researchers evaluated several widely used machine learning models for detecting Man-in-the-Middle attacks in smart grid industrial control systems, using the RESLab smart grid dataset. Fast-gradient sign method perturbations reduced Random Forest detection accuracy by 17.48 percentage points for binary classification and 14.86 percentage points for multiclass classification. Retraining the Random Forest classifier with adversarial samples raised accuracy under fixed-source FGSM attacks to 81.59% (binary) and 81.31% (multiclass) while preserving performance on clean data.

This structured narrative review synthesizes attacks, defenses, threat models and evaluation practices for retrieval-augmented generation (RAG) systems in work published between 2020 and September 2026, organized by pipeline stage from ingestion through tool execution. It finds that poisoning, extraction, membership inference and indirect prompt injection are distinct but composable threats, and that layered controls spanning provenance, access control, retrieval-time anomaly detection, context isolation, output monitoring and tool permissions are needed. The paper was published in Electronics on 2026-11-11.

A research post from an outside group says it was surprised by a passage in the Gemini 4 safety report indicating that some RL training components observe thoughts as context and up-weight reasoning on track to good outputs. The authors say the passage is unclear, that they assumed it meant auxiliary rewards scoring partial chains of thought, and that GDM staff and a blog post have given conflicting accounts. They argue such CoT training could suppress misaligned verbalizations and reduce CoT faithfulness.

Researchers benchmarked FlyNet, a task-optimized recurrent network constrained by the fly visual system's synaptic connectome, against CNNs, RAFT and CTRNN models on a collision-detection task. FlyNet is adversarially vulnerable but more robust to common video corruptions than RAFT relative to compute, using up to five orders of magnitude fewer FLOPs. Adversarial textures and the reverse-phi illusion disrupt the same low-level motion statistic, suggesting a shared circuit vulnerability.

The study compares three evaluation frameworks (trace-based LLM judging, text-only LLM judging with red-teaming, and deterministic heuristics) on two enterprise analytics agents using frozen execution traces. On 72 ground-truth cases, the trace-based framework reached the best balance (F1=0.90), while text-only evaluation matched its recall but produced 5x more false alarms. Phase 3 found 24 novel failures with zero overlap between functional testing and security red-teaming.

The paper introduces SEAL, a sparse adversarial protection framework that modifies only 5% of spatial locations in a facial image to stop diffusion personalization from learning an individual's identity. It jointly optimizes perturbation locations and values through a differentiable top-k masking scheme, targeting the frozen VAE encoder's latent distribution. On CelebA-HQ and VGGFace2, it reports over 41 dB PSNR, 0.97 SSIM and at most 0.07 LPIPS, with identity similarity reduced to at most 0.15 after personalization.

GARD is a post-hoc defense for backdoored multimodal contrastive encoders such as CLIP that needs no knowledge of the attack target or trigger. It optimizes complementary global and local adversarial probes to displace clean-image representations, then realigns them with detached clean anchors while preserving image-text alignment. Across six visual backdoor attacks and two CLIP visual backbones, it reduces attack success rate below 1% in all 12 combinations while keeping competitive clean accuracy.

Adversarial training often produces "pseudo-attacks" that raise the loss without truly altering a model's confidence structure. The authors propose a hybrid objective combining cross-entropy and KL divergence, plus perturbations that maximize distribution distance on the confidence manifold, with the optimal direction aligned to principal eigenvectors of the Fisher information matrix. Experiments across multiple datasets and architectures report better adversarial robustness and transferability, especially against structural perturbations in high-confidence regions.

This paper argues that security in agentic coding harnesses is a structural property of the cognitive architecture and permission composition, not of individual models. It presents ActiveSecHarness, a neurosymbolic reference-monitor design using HMAC capability tokens, a two-dimensional taint lattice, and deterministic Ring-0 guards. On 252 fixtures, Ring 0 reports ABR 1.000 and FPR 0.000 in 12.7 μs, versus a local LLM judge that refuses 98.3% of benign work at 142.3 ms.

Researchers examine whether acoustic side-channel attacks on keyboards, which recover typed text from keystroke sounds captured by a commodity microphone, still work outside idealized lab conditions. The study addresses unseen keyboards, diverse users, noisy environments and compressed transmission channels. The authors report that inference survives these conditions when the acoustic model is made explicitly domain-invariant and its noisy predictions are corrected by a language model.

OpenAI researchers study how pretraining, architecture, reinforcement learning and model-generated data affect two proxies for chain-of-thought monitorability: no-CoT capability and CoT controllability. They report that no-CoT capability rises smoothly as base model perplexity falls, and that CoT controllability increases during RL even without process supervision beyond length constraints and final answer grading.

This research paper examines memory poisoning attacks on the semantic memory of LLM-based agents in multi-agent systems, where malicious agents can alter an agent's knowledge and behavior. The authors propose private knowledge retrieval and private inference, with a signature system on the knowledge base to block undesired manipulation. Each agent runs a local Prolog-like inference system over Horn clauses that queries an externally signed knowledge base privately. The authors provide Python implementations using k-anonymity and information-theoretical privacy (IT-PIR).

New research from OpenAI and Upwork examines how AI is changing freelance demand, using ChatGPT usage data and Upwork job postings from 2023 to 2026. Some services are losing market share on Upwork, while AI-related projects combine tasks in new ways and carry larger budgets and more hours. Clients increasingly request AI across many kinds of work, and AI-related postings are 45% more diverse in task mix between early 2023 and late 2025.

MOAI is a homomorphic encryption framework for non-interactive secure transformer inference, so LLM inference can run directly over encrypted inputs in cloud deployments. It combines column and diagonal packing across all layers and uses rotation-free algorithms for Softmax and LayerNorm, removing 2,448 HE rotations in BERT-base inference. The authors report a 52.8% reduction in evaluation time versus THOR and a 55.7% reduction versus Powerformer, at 2.36 minutes per input on a single GPU, and the implementation is open source.

The paper proposes Target-free Entropy-guided Residual Modulation (TERM), an inference-time defense against jailbreak attacks on LLMs. TERM calibrates attention entropy against each prompt's own prefill baseline and uses adaptive scaling to bound changes to the model's residual updates, without a predefined safety direction. Tested against seven defenses, seven jailbreak attacks and two over-refusal benchmarks on Llama, Qwen and Gemma models (Qwen3 from 0.6B to 32B), it improves jailbreak robustness with low over-refusal and modest inference overhead.

This paper proposes Target-free Entropy-guided Residual Modulation (TERM), an inference-time defense against jailbreak attacks on LLMs. TERM calibrates attention entropy against each prompt's own prefill baseline and uses the result to bound-scale the model's own residual updates, without a predefined safety direction. The authors report improved jailbreak robustness with low over-refusal across Llama, Qwen, and Gemma models from 0.6B to 32B parameters, with modest inference overhead.

Researchers present Talos, an inline gate that judges agent requests to industrial controllers in milliseconds and redirects flagged agents to a mirror-twin decoy of the real plant. Evaluated against 2,400 autonomous attacker agents across four tiers of escalating concealment, no write reached the real plant in any tier. The pipeline also extracts attacker intelligence, with identity clues such as self-reported toolchain fingerprints and targeted register lists accumulating into a high-confidence profile.