AI Sec Watch

Anthropic is testing a new AI model called Mythos that has advanced cybersecurity capabilities but also poses security risks, causing the company to plan a slow rollout. The announcement led to significant stock price drops for major cybersecurity companies, as investors worry that powerful AI tools could make hacking easier and disrupt the cybersecurity industry.

Open WebUI's speech-to-text endpoint has a path traversal vulnerability where an authenticated user can craft a malicious filename to trigger an error that leaks the server's absolute file path. The vulnerability exists because the code doesn't sanitize the filename before using it in a file operation, unlike similar upload handlers elsewhere in the codebase.

AI Code has a feature that automatically runs terminal commands (direct instructions to a computer's operating system) if it thinks they're safe, but an attacker can use prompt injection (tricking an AI by hiding instructions in its input) to disguise malicious commands as safe ones, causing them to execute without user approval.

A path traversal vulnerability (a security flaw where attackers manipulate file paths to access files they shouldn't) exists in the awesome-llm-apps project's Beifong AI News and Podcast Agent backend. An unauthenticated attacker can exploit this weakness in the stream-audio endpoint to read arbitrary files from the server, potentially exposing sensitive data like configuration files and credentials.

This article briefly mentions several security-related news items including a Heritage Bank data breach, a new State Department cyber threat unit, and LA Metro disruptions, along with stories about a Palo Alto recruiter scam, an anti-deepfake chip (technology designed to detect AI-generated fake videos), and Google's quantum computing deadline for 2029. The content provided is minimal and does not go into detail about any of these incidents.

A Dutch court has ordered Elon Musk's xAI and its chatbot Grok to stop creating non-consensual AI-generated sexual images of adults and children, with daily fines of 100,000 euros for non-compliance. The ruling came after the non-profit Offlimits reported that Grok generated an estimated three million sexualized images in about two weeks, including over 23,000 depicting children, and found that xAI's previous restrictions on creating such images were easily bypassed. The case adds to mounting legal pressure on xAI, with investigations underway in Europe and lawsuits filed in the United States.

OpenAI has started a bug bounty program, which is a system where security researchers can report problems and receive rewards for finding them. The program focuses on design or implementation issues (flaws in how the AI is built or how it works) that could cause serious harm through misuse or safety problems.

Wikipedia has banned the use of LLMs (large language models, the AI systems behind tools like ChatGPT) for generating or rewriting article content, as the site's volunteer editors voted that AI often violates Wikipedia's core principles. Two exceptions allow AI for translations and minor copy edits to editors' own writing, though Wikipedia cautions that LLMs can accidentally change meaning or add unsupported information beyond what was requested.

Researchers discovered a new backdoor attack (a security flaw where hidden malicious code is planted in training data) on Graph Neural Networks, or GNNs (AI models designed to understand interconnected data). The attack uses a single trigger node (a specially crafted fake data point) attached to a target node to trick the GNN into making wrong predictions not just on that node, but also on its immediate neighbors, while remaining stealthy and achieving over 95% success rates even against existing defenses.