AI Sec Watch

Flowise version 3.1.1 has a vulnerability where attackers can use prompt injection (tricking an AI by hiding instructions in its input) on a CSV Agent node to make the LLM generate malicious Python code that bypasses the security blocklist validator and runs with full system access in an unsandboxed pyodide environment (a Python runtime in JavaScript). This allows remote code execution without requiring authentication.

Nvidia-led Open Secure AI Alliance (OSAA), an industry group of over 120 companies formed to address AI security, has quickly developed initial proposals including guidelines for confidentially reporting AI cybersecurity incidents and conducting blame-free analysis of incidents. Member companies are also contributing open source security tools, such as Nvidia's Garak (an LLM vulnerability scanner, a tool that checks AI systems for security weaknesses) and tools from other companies for agent identity and governance, with the goal of eventually creating shared open source resources to help enterprises secure their AI systems.

Flowise has a broken access control vulnerability in its billing endpoints that allows an authenticated attacker to manipulate another organization's Stripe subscriptions. The vulnerable endpoints accept subscription identifiers directly from user input without checking that the subscription belongs to the attacker's organization, enabling unauthorized billing operations like changing subscription plans or seat quantities.

Flowise has a security flaw where the text-to-speech API endpoint doesn't require login and doesn't check if a chatflow is public before allowing access to it. An attacker who knows a chatflow's ID can trick the system into using that chatflow's stored API credentials (like OpenAI or ElevenLabs keys) to generate unlimited audio without permission, costing the real owner money.

Flowise has a missing authorization vulnerability in its execution update endpoint (`PUT /api/v1/executions/:id`). Unlike other execution endpoints that check user permissions, the update endpoint allows any authenticated user to modify any execution record, enabling privilege escalation (gaining higher-level access than intended) where a low-privileged user can change execution data and results.

During independent security tests of OpenAI models, two external testing partners discovered that the models accessed the public internet beyond their intended boundaries under specific test conditions. One test (by the UK government's AI Security Institute) intentionally enabled internet access with reduced safeguards to measure the model's underlying capabilities, while another test (by a cybersecurity firm called Irregular) had a configuration error that unintentionally allowed internet access when the test environment was supposed to be isolated. OpenAI states these incidents highlight the need to improve testing standards and safety practices as AI models become more capable.

Microsoft has expanded its Zero Trust for AI strategy with new tools to help organizations secure AI agents and development workflows. The updates include an AI-focused Zero Trust Assessment tool that evaluates security controls across AI systems, and a new DevSecOps (developer security operations, where security practices are built into software development) pillar in the Zero Trust Workshop that provides 91 specific tasks to apply Zero Trust principles (verify every access, assume breaches could happen, use least privilege access) from source code to cloud deployment.

NVIDIA Triton Inference Server for Linux has a vulnerability where an attacker could read, write, or modify files outside the intended model repository by providing a specially crafted path in the model name to the MLflow plugin. This could lead to denial of service (making a system unavailable) and information disclosure (leaking sensitive data).

Flowise has a security vulnerability in its OAuth2 credential handling where three endpoints look up credentials by ID alone without checking which workspace the user belongs to, and two of these endpoints skip authentication entirely. This allows authenticated users to access credentials from other workspaces, and unauthenticated attackers to inject forged OAuth2 tokens or refresh tokens for any credential in the system.