AI Sec Watch

Large Vision Language Models (VLMs, which are AI systems that process both images and text) are vulnerable to jailbreak attacks (attempts to trick the AI into ignoring its safety guidelines). VLM-Guard is a detection framework that identifies and monitors a small set of neurons (individual computational units, about 0.2% of the total) that are linked to unsafe behavior, allowing it to catch jailbreak attempts without requiring model fine-tuning (adjusting the AI's internal parameters through additional training). The approach is lightweight and effective at detecting attacks while maintaining normal performance on safe inputs.

Federated learning (a method where multiple computers train an AI model together without sharing their raw data) is vulnerable to poisoning attacks, where malicious participants sabotage the shared model. This paper proposes SpecShield, a defense that proactively tests each participant's model using carefully crafted perturbations (small, intentional changes) and analyzes their responses using frequency-domain analysis (a mathematical technique that examines patterns at different scales) to distinguish malicious clients from honest ones.

Ollama versions before 0.17.1 have a heap out-of-bounds read vulnerability (a bug where code reads memory outside its intended boundaries) in the GGUF model loader (the component that loads GGUF files, a machine learning model format). An attacker can upload a malicious GGUF file through the /api/create endpoint (an unprotected interface) with fake tensor size information, causing the server to read beyond the file's actual data and leak sensitive information like API keys and user conversations, which can then be stolen through the /api/push endpoint.

This writeup describes vulnerabilities found in Microsoft Copilot products that allow attackers to steal sensitive data through multiple attack chains, including data exfiltration via HTML preview features, hijacking the AI's long-term memory through prompt injection (tricking an AI by hiding instructions in its input), and creating persistent backdoors. The vulnerabilities, assigned CVE-2026-24299, exploited what researchers call the "lethal trifecta," where an AI has access to private data, untrusted content, and external communication channels simultaneously.

Security agencies including CISA have issued joint guidance on safely deploying agentic AI (autonomous AI systems that can take actions independently), warning that prompt injection (tricking an AI by hiding instructions in its input) and other attacks are common threats. The advisory recommends organizations implement strict access controls using the principle of least privilege (giving systems only the minimum permissions they need), continuous monitoring with human oversight, and careful testing before deploying AI agents to production environments.

OpenAI has introduced Advanced Account Security, an optional feature for ChatGPT users at high risk of targeted attacks, such as journalists and political dissidents. The feature strengthens account protection by disabling password-based login in favor of physical security keys or passkeys, replacing email and SMS account recovery with backup passkeys and recovery keys, shortening sign-in sessions, and automatically excluding user conversations from AI model training.

Fake IT workers, increasingly enabled by AI tools and deepfakes, are being hired into organizations as an insider threat (a risk posed by trusted employees or contractors with system access). State actors like North Korea and individuals use stolen or synthetic identities, AI-assisted interview responses, and social engineering to bypass recruitment screening and gain access to sensitive systems and data.

OpenAI rearchitected its WebRTC (web real-time communication, a standard protocol for sending low-latency audio and video between clients and servers) infrastructure to handle voice AI at scale while maintaining natural conversation speed. The team addressed three constraints that conflicted at scale: one-port-per-session media termination, stateful ICE (Interactive Connectivity Establishment, the process for establishing connections across firewalls) and DTLS (Datagram Transport Layer Security, encryption for real-time data) session stability, and global routing latency. OpenAI built a new split relay plus transceiver architecture that preserves standard WebRTC behavior for users while changing how data packets are routed internally.

The US Pentagon has signed contracts with seven tech companies (Google, Microsoft, Amazon Web Services, Nvidia, OpenAI, Reflection, and SpaceX) to use their AI systems on classified military networks to help with battlefield decisions and operations. However, concerns remain about potential risks, including privacy invasion, civilian casualties, and over-reliance on AI without proper human oversight, with questions still being worked out about appropriate levels of human involvement and operator training.