AI Sec Watch

A TOCTOU attack (time-of-check to time-of-use, a type of race condition where a system checks something and then uses it, but the situation changes in between) can trick AI agents that control computers by changing what's on the screen while the AI is thinking. For example, an attacker can swap out a button with a different one, or overlay a fake button on top of a real one, so the AI clicks something it didn't intend to, like sending an email or visiting a malicious site.

This academic survey examines hallucinations in large visual and language models, which are instances where AI systems generate false or nonsensical information that appears plausible. The paper, published in ACM Computing Surveys in October 2026, provides a comprehensive overview spanning 36 pages of research on this problem affecting both language models (AI systems trained on text) and multimodal models (AI systems that process both images and text).

A research paper shows that large language models (LLMs) are vulnerable to prompt injection attacks (tricks where attackers hide malicious instructions in text input) because they rely on role tags (formatting markers that separate different instruction blocks) as their main security mechanism, but these tags don't actually reflect how the model processes information internally. The researchers conclude that unless LLMs develop a genuine ability to understand and maintain role boundaries, prompt injection attacks will remain difficult to prevent permanently.

CIOs face pressure to rapidly adopt AI across their organizations to prove business value, but must balance this speed with managing new security and governance risks. AI introduces unique challenges because its behavior is indeterminate (unpredictable and hard to verify like traditional technology) and employees are eager to use it without oversight, creating what's called shadow use (unauthorized use of tools that bypasses IT controls). Organizations should clarify their specific business goals and conduct a risk assessment before implementing AI rather than adopting it out of fear of falling behind.

A new malware called Gaslight, created by North Korea-aligned hackers, targets macOS systems and uses prompt injection (tricking an AI by hiding instructions in its input) to disrupt AI tools that analyze malware. The malware embeds fake system-failure messages designed to confuse AI-assisted analysis tools, while also stealing sensitive data like browser histories and passwords through a command-and-control (C2, a server that lets attackers remotely control infected computers) channel powered by Telegram.

Anthropic, a major AI company, is rapidly expanding its data center operations in Asia-Pacific by hiring 13 people, with eight positions in Australia and Japan, to handle increasing demand for its AI products. The company is building infrastructure in these regions because they offer advantages like renewable energy, political stability, and security benefits, though Australia's copyright laws present a potential obstacle to large-scale expansion.

Agentic AI (AI systems that can work independently on tasks for extended periods, rather than just answering single questions) is transforming how people work by handling longer, more complex tasks instead of short interactions. At OpenAI, a tool called Codex shifted from being used mainly by engineers to becoming the primary AI tool across all departments, including non-technical ones like Legal and Recruiting, with usage growing dramatically over the past year. Users increasingly delegate tasks that would take humans hours or even days to complete, with some users running dozens of hours of parallel agent tasks in a single day.

Anthropic accused Alibaba of carrying out a massive distillation attack (a method where a weaker AI model is trained using outputs from a stronger model) involving 28.8 million exchanges with fraudulent accounts between April and June 2024. Anthropic called for coordinated government and industry action to combat illegal distillation, though Alibaba has not yet responded to the allegations.

Twenty, an open-source CRM platform, had a vulnerability before version 2.9.0 where authenticated users could access other workspaces' AI agent data through IDOR (insecure direct object reference, a flaw where the system doesn't verify that requested data belongs to the user). Attackers with access to a workspace could view other users' chat histories, tool calls, and outputs by knowing their agent or turn IDs, which were visible in the settings page URL.