AI Sec Watch

The European Commission is hiring AI specialists to work in the AI Office, which will enforce the EU's AI Act by overseeing compliance of general-purpose AI models (large AI systems available to the public). The office will have real regulatory powers to require companies to implement safety measures, restrict models, or remove them from the market, and will develop evaluation tools and benchmarks to identify dangerous AI behaviors.

CVE-2024-1727 is a CSRF vulnerability (cross-site request forgery, where an attacker tricks a victim into making unintended requests) in Gradio that lets attackers upload large files to a victim's computer without permission. An attacker can create a malicious webpage that, when visited, automatically uploads files to the victim's system, potentially filling up their disk space and causing a denial of service (making the system unusable).

The European AI Office is a new EU regulator created to oversee general purpose AI (GPAI) models and systems, which are AI systems designed to perform a wide range of tasks, across all 27 EU Member States under the AI Act. It monitors compliance, analyzes emerging risks, develops evaluation capabilities, produces voluntary codes of practice for companies to follow, and coordinates enforcement between national regulators and international partners. The Office also supports small and medium businesses with compliance resources and oversees regulatory sandboxes, which are controlled environments where companies can test AI systems before full deployment.

A vulnerability in datahub-helm (Helm charts, which are templates for deploying applications on Kubernetes clusters) versions 0.1.143 through 0.2.181 allowed personal access tokens (credentials that grant access to the system) to be created using a publicly known default secret key instead of a random one. This meant attackers could potentially generate their own valid tokens to access DataHub instances if Metadata Service Authentication (a security feature) was enabled during a specific vulnerable time period.

Moby (the container framework underlying Docker) has a bug in how it handles DNS requests from internal networks (networks isolated from external communication). When a container on an internal network needs to resolve a domain name, Moby forwards the request through the host's network namespace instead of the container's own network, which can leak data to external servers that an attacker controls. Docker Desktop is not affected by this issue.

NextChat (also called ChatGPT-Next-Web) version 2.11.2 and earlier has two security flaws: SSRF (server-side request forgery, where attackers trick the server into making unwanted requests) and XSS (cross-site scripting, where attackers inject malicious code into web pages). These flaws let attackers read internal server data, make changes to it, hide their location by routing traffic through the app, or attack other targets on the internet.

CVE-2024-27565 is a server-side request forgery (SSRF, a flaw that allows attackers to trick a server into making unwanted requests to other systems) vulnerability found in the weixin.php file of ChatGPT-wechat-personal at commit a0857f6. This vulnerability lets attackers force the application to make arbitrary requests on their behalf. The vulnerability has a CVSS 4.0 severity rating (a moderate score on a 0-10 scale measuring how serious a security flaw is).

LangChain versions up to 0.1.10 have a path traversal vulnerability (a flaw where an attacker can use ../ sequences to access files outside the intended directory) that allows someone controlling part of a file path to load configurations from anywhere instead of just the intended GitHub repository, potentially exposing API keys or enabling remote code execution (running malicious commands on a system). This bug affects how the load_chain function handles file paths.

Attackers can create conditional prompt injection attacks (tricking an AI by hiding malicious instructions in its input that activate only for specific users) against Microsoft Copilot by leveraging user identity information like names and job titles that the AI includes in its context. A researcher demonstrated this by sending an email with hidden instructions that made Copilot behave differently depending on which person opened it, showing that LLM applications become more vulnerable as attackers learn to target specific users rather than all users equally.