AI Sec Watch

In MLflow (a machine learning tool for managing experiments), when basic authentication is enabled, certain endpoints that show trace information (a record of how the AI made decisions) and allow users to assess traces are not properly checking user permissions. This means any logged-in user can view traces and create assessments even if they shouldn't have access to them, risking exposure of sensitive information and unauthorized changes.

Open WebUI has an insecure direct object reference (IDOR, a flaw where an app doesn't properly check if a user should access specific data) in its retrieval API that lets any authenticated user read other users' private memories and uploaded files by guessing collection names like 'user-memory-{USER_UUID}' or 'file-{FILE_UUID}'. The vulnerability exists because the API checks that a user is logged in, but doesn't verify they own the data they're requesting.

Open WebUI's file batch processing endpoint lacks an ownership check, allowing any authenticated user to overwrite files in shared knowledge bases by knowing their IDs. An attacker can then poison the RAG (retrieval-augmented generation, where an AI pulls in external documents to answer questions) system, causing the LLM to serve the attacker's malicious content to other users.

Anthropic is testing a new AI model called Mythos that has advanced cybersecurity capabilities but also poses security risks, causing the company to plan a slow rollout. The announcement led to significant stock price drops for major cybersecurity companies, as investors worry that powerful AI tools could make hacking easier and disrupt the cybersecurity industry.

Open WebUI's speech-to-text endpoint has a path traversal vulnerability where an authenticated user can craft a malicious filename to trigger an error that leaks the server's absolute file path. The vulnerability exists because the code doesn't sanitize the filename before using it in a file operation, unlike similar upload handlers elsewhere in the codebase.

AI Code has a feature that automatically runs terminal commands (direct instructions to a computer's operating system) if it thinks they're safe, but an attacker can use prompt injection (tricking an AI by hiding instructions in its input) to disguise malicious commands as safe ones, causing them to execute without user approval.

A path traversal vulnerability (a security flaw where attackers manipulate file paths to access files they shouldn't) exists in the awesome-llm-apps project's Beifong AI News and Podcast Agent backend. An unauthenticated attacker can exploit this weakness in the stream-audio endpoint to read arbitrary files from the server, potentially exposing sensitive data like configuration files and credentials.

This article briefly mentions several security-related news items including a Heritage Bank data breach, a new State Department cyber threat unit, and LA Metro disruptions, along with stories about a Palo Alto recruiter scam, an anti-deepfake chip (technology designed to detect AI-generated fake videos), and Google's quantum computing deadline for 2029. The content provided is minimal and does not go into detail about any of these incidents.

A Dutch court has ordered Elon Musk's xAI and its chatbot Grok to stop creating non-consensual AI-generated sexual images of adults and children, with daily fines of 100,000 euros for non-compliance. The ruling came after the non-profit Offlimits reported that Grok generated an estimated three million sexualized images in about two weeks, including over 23,000 depicting children, and found that xAI's previous restrictions on creating such images were easily bypassed. The case adds to mounting legal pressure on xAI, with investigations underway in Europe and lawsuits filed in the United States.