AI Sec Watch

Flowise, an AI tool, has a hardcoded setting that allows any webpage on the internet to make requests to its text-to-speech (TTS, a feature that converts written text into spoken audio) endpoint using your stored credentials. This bypasses the server's normal cross-origin request protection (CORS, which controls what websites can access a server's data), letting malicious webpages secretly generate speech on your behalf.

Google announced Gemini Spark, an upcoming AI agent product that connects with Google apps like Gmail and Drive, which runs on Gemini 3.5 Flash and a tool called Antigravity. To address prompt injection risks (tricking an AI by hiding instructions in its input), Google stated that Spark operates in isolated virtual environments with encrypted credentials, data loss prevention policies, and a secure gateway, though the author expresses concern about whether these protections are sufficient given the sensitive data users may process through it.

The `diffusers` package has a TOCTOU (time-of-check-time-of-use, where a security check happens at one moment but the actual data used comes from a different moment) vulnerability in its `DiffusionPipeline.from_pretrained` function that loads models from HuggingFace Hub. An attacker can bypass the `trust_remote_code` security check by updating a repository between two separate download calls, allowing arbitrary code to execute without the user explicitly approving it.

RTK (Rust Token Killer, a tool that filters sensitive data before showing command output to an LLM) had a vulnerability where it automatically loaded filter configuration files from a project directory without asking the user first, allowing attackers to secretly modify what an LLM sees. An attacker could place a malicious filter file in a repository to hide or alter command output (like file contents or security scan results) without any warning, potentially concealing malicious code during development.

A UK study by the thinktank Demos found that AI chatbots like ChatGPT gave voters false information in response to 34% of questions about the Scottish election, including made-up scandals and invented candidates. The Electoral Commission has called for new legal controls to regulate AI platforms and prevent this kind of misinformation (false information spread to deceive people).

AI, particularly agentic AI (AI systems that can plan and take actions independently), is making attacks on applications faster, cheaper, and more widespread than ever before. Attackers are now targeting all apps as primary threats rather than some being less important, and apps face hostile attacks within hours of being published online instead of days. Critical sectors like medical devices and automotive apps are seeing the steepest increases in attacks because AI tools have made it easier to reverse engineer (understand how software works by analyzing it) the complex, specialized code that once protected these systems.

Simply using security and privacy benchmarks (standardized tests that measure how well a system performs) is not enough to ensure AI is truly secure, because benchmarks don't accurately measure AI capabilities. Instead of relying on benchmarks alone, organizations should apply proven security engineering practices, such as process-driven standards like BSIMM (Building Security In Maturity Model, a framework that guides companies through security best practices), while staying extra vigilant since AI systems don't have a single reliable security measurement like software does.

Google and other companies are expanding AI labeling systems like SynthID (invisible watermarking that tags AI-generated images) and C2PA Content Credentials to help people identify fake or AI-generated content online. These technologies aim to combat deepfakes (manipulated videos or images made to look realistic) and other misleading AI content that has been deceiving people on social media.

1Password partnered with OpenAI to protect credentials from being leaked by AI coding agents, which are AI systems that can write and deploy software automatically. The companies created an Environments MCP Server (a module that connects different systems together) for Codex that gives AI agents access to credentials only when needed, without storing them in code, prompts, or the AI model's memory where they could be stolen. Credentials are issued just-in-time, scoped to specific tasks, and kept encrypted in 1Password's vault rather than exposed where attackers could find them.