AI Sec Watch

The OWASP Generative AI Security Project, an organization focused on application security, announced nine new corporate sponsors to support efforts in improving security for generative AI technologies. The sponsors, including companies like ByteDance and Trend Micro, represent increased investment and momentum in making AI systems more secure.

PyTorch 2.6.0 contains a vulnerability in the torch.nn.functional.ctc_loss function (a component used for speech recognition tasks) that can cause denial of service (making the system unavailable). The vulnerability requires local access to exploit and has been publicly disclosed, though its actual existence is still uncertain.

A critical vulnerability (CVE-2025-3677) was found in lm-sys FastChat version 0.2.36 and earlier in the file apply_delta.py. The flaw involves deserialization (converting data back into code or objects, which can be dangerous if the data comes from an untrusted source) and can only be exploited by someone with local access to the affected system.

Mattermost (a team communication platform) versions 10.4.2 and earlier, 10.5.0 and earlier, and 9.11.9 and earlier don't properly block which websites their built-in AI tool can contact. This allows logged-in users to use prompt injection (tricking the AI by hiding instructions in their input) to steal data from servers that the Mattermost system can access.

The AI Safety Newsletter highlights the launch of AI Frontiers, a new publication featuring expert commentary on critical AI challenges including national security risks, resource access inequality, risk management approaches, and governance of autonomous systems (AI agents that can make decisions without human input). The newsletter presents diverse viewpoints on how society should navigate AI's wide-ranging impacts on jobs, health, and security.

In Aidex versions before 1.7, a logged-in attacker could exploit an open registry to run unauthorized commands on the system through prompt injection attacks (tricking the AI by hiding malicious instructions in user input) via the chat message endpoint. This allowed them to execute operating system commands, access databases, and invoke framework functions.

MaxKB (Max Knowledge Base) is an open source system that answers questions using a large language model and RAG (retrieval-augmented generation, where an AI pulls in external documents to answer questions). A reverse shell vulnerability (a security flaw that lets attackers gain control of a system remotely) exists in its function library module and can be exploited by privileged users to create unauthorized access.

BentoML is a Python library for building AI model serving systems, but versions before 1.4.8 had a vulnerability in its runner server that allowed attackers to execute arbitrary code (unauthorized commands) by sending specially crafted requests with specific headers and parameters, potentially giving them full access to the server and its data.

Spammers used OpenAI's GPT-4o-mini model to generate unique spam messages for each target website, allowing them to bypass spam-detection filters (systems that block unwanted messages) across over 80,000 sites in four months. The spam campaign, called AkiraBot, automated message delivery through website contact forms and chat widgets to promote search optimization services. OpenAI revoked the spammers' account in February after the activity was discovered.