AI Sec Watch

At Google I/O, DeepMind CEO Demis Hassabis announced that Google aims to use AI to transform drug discovery and eventually solve all diseases. The article appears to be a critical analysis examining the feasibility and implications of this ambitious claim.

AI agents (autonomous systems using LLMs to solve problems) create security risks because LLMs are unpredictable and vulnerable to prompt injection (tricking an AI by hiding instructions in its input), so they can make harmful decisions with confidence. The solution is to place authorization controls (decisions about which actions are allowed) at the boundary where the agent calls external tools, rather than relying on hard-coded workflows or human approval alone. Amazon Bedrock AgentCore uses Cedar, an open-source authorization policy language, to centralize and enforce these controls outside the LLM where they cannot be bypassed.

OpenAI and Anthropic's expected IPO valuations (both projected over $800 billion) depend on maintaining high pricing power, but cheaper AI alternatives are rapidly emerging and becoming competitive. Chinese AI labs like DeepSeek and Kimi charge a fraction of what OpenAI and Anthropic do for comparable work, and enterprises are adopting cost-reduction strategies like "advisor models" (where a cheap open-source model handles most tasks and only calls expensive frontier models when needed), causing usage of Chinese models on some platforms to jump from 1% to 60% in just one year.

OpenAI is preparing to confidentially file documents for an IPO (initial public offering, when a private company becomes publicly traded) as soon as this week, working with major investment banks like Goldman Sachs and Morgan Stanley. The company, valued at over $850 billion, is planning this public debut as part of normal strategic planning, though leadership hasn't confirmed a specific timeline.

Microsoft released two open-source tools to help developers test AI agent security during development. RAMPART is a testing framework (built on PyRIT, an earlier tool) that lets developers write test cases to find safety problems like cross-prompt injections (when untrusted data reaches an AI indirectly through sources like emails or files) and data exfiltration (unauthorized data leakage). Clarity is a planning tool that guides developers through design decisions early in a project, before coding begins, so potential issues can be addressed cheaply rather than fixed later.

Google has introduced a new YouTube Shorts Remix feature that uses Gemini (Google's AI model) to let users restyle or modify other people's videos. Users can transform clips into different art styles like pixel art or anime, or digitally alter content by changing appearances, adding people, or inserting themselves into videos. Creators can choose whether to allow or block others from remixing their videos.

Google is integrating its Gemini AI model into search ads, which will now display recommended products with AI-generated explanations of why you should buy them. This update is part of Google's broader shift toward AI-powered search results, including a new conversational search box and AI-generated content alongside traditional search results.

GitHub confirmed that attackers compromised an employee's device through a poisoned VS Code extension (a malicious add-on program for a code editor), leading to the theft of code from around 3,800 internal repositories. The breach was detected and contained quickly, and GitHub is investigating the incident while validating that no customer data was affected, only internal GitHub code.

Flowise has a security flaw in its `/api/v1/chatflows/apikey` endpoint that allows a user with a valid API key to view chatflow configurations (including system prompts, workflow graphs, and credential IDs) from other workspaces, as long as those chatflows don't have an API key assigned. The endpoint returns both the user's own chatflows and all unprotected chatflows across the entire system without filtering by workspace, breaking the isolation between workspaces.