AI Sec Watch

Researchers at Google DeepMind have identified a vulnerability called 'AI Agent Traps' that allows attackers to manipulate and exploit AI agents (autonomous programs that can browse the web and take actions) by hosting malicious web content designed to deceive them. This research maps out how these attacks work against AI systems that visit websites.

Healthcare workers are increasingly using AI tools on their own to handle heavy workloads, and organizations cannot stop this trend. The source emphasizes that healthcare organizations should strengthen their security practices to reduce the damage if these unsanctioned AI tools are compromised or misused.

OWASP (Open Web Application Security Project, a standards group for security best practices) has updated its generative AI security guidance to address 21 identified risks in AI systems. The update recommends that companies use separate but coordinated defense strategies tailored specifically for generative AI (AI that creates text, images, or code) and agentic AI (AI that can take actions independently).

OpenAI is launching a Safety Fellowship program (September 2026 to February 2027) for external researchers to conduct independent studies on safety and alignment (making sure AI systems behave as intended and don't cause harm) of advanced AI systems. Fellows will work on topics like safety evaluation, ethics, robustness, privacy protection, and oversight of AI agents, receiving mentorship, compute resources, and a monthly stipend while producing research outputs like papers or datasets.

Attackers are increasingly exploiting legitimate AI systems and services instead of using traditional malware, a trend called "living off the AI land." Examples include poisoning MCP servers (tools that connect AI assistants to external services) in supply chains, abusing AI platforms like Claude and Copilot as command-and-control channels (hidden pathways for sending malicious instructions), and hijacking AI agents (automated systems that perform tasks) to extract sensitive data or perform destructive actions. The shift represents a fundamental change in AI security threats, moving beyond simple prompt injection (tricking an AI by hiding instructions in its input) to more sophisticated agent hijacking (taking control of automated AI systems).

Commercial off-the-shelf software (COTS, meaning ready-made software products sold online or in stores) initially seems attractive because it deploys quickly and costs less than custom development, but organizations often get trapped when they want to switch platforms, as their systems become deeply entangled with the vendor's technology. AI-powered security tools are creating a new type of lock-in by relying on proprietary training data, vendor-specific threat intelligence feeds (collections of indicators showing cyber attacks), and specialized hardware, making it expensive and difficult to migrate away.

OpenClaw, an open-source AI assistant built by an Austrian developer, sparked a major trend in China in March 2024 because it can be customized to work with Chinese AI models, unlike Western tools like ChatGPT that are inaccessible there. Users enthusiastically adapted OpenClaw's code to create personalized versions they called "lobsters," using them for tasks like e-commerce product listings, stock analysis, and productivity, with some claiming dramatic efficiency gains. The phenomenon reflects China's broader push to develop and embrace AI technology, driven by government support and the success of homegrown platforms like DeepSeek.

Google has integrated Gemini (an AI assistant that's built into Google services) into Google Maps, allowing it to help plan daily itineraries by suggesting nearby locations. The author tested this feature by having Gemini plan a full day around their city and found it effective, discovering both obvious and unexpected recommendations for places to visit.

A vulnerability (CVE-2026-5530) has been discovered in Ollama up to version 18.1 that allows attackers to perform SSRF (server-side request forgery, where an attacker tricks a server into making unwanted requests on their behalf) through the Model Pull API component. The flaw can be exploited remotely by authenticated users, and the vendor has not responded to disclosure attempts.