AI Sec Watch

NextChat (also called ChatGPT-Next-Web) version 2.11.2 and earlier has two security flaws: SSRF (server-side request forgery, where attackers trick the server into making unwanted requests) and XSS (cross-site scripting, where attackers inject malicious code into web pages). These flaws let attackers read internal server data, make changes to it, hide their location by routing traffic through the app, or attack other targets on the internet.

CVE-2024-27565 is a server-side request forgery (SSRF, a flaw that allows attackers to trick a server into making unwanted requests to other systems) vulnerability found in the weixin.php file of ChatGPT-wechat-personal at commit a0857f6. This vulnerability lets attackers force the application to make arbitrary requests on their behalf. The vulnerability has a CVSS 4.0 severity rating (a moderate score on a 0-10 scale measuring how serious a security flaw is).

LangChain versions up to 0.1.10 have a path traversal vulnerability (a flaw where an attacker can use ../ sequences to access files outside the intended directory) that allows someone controlling part of a file path to load configurations from anywhere instead of just the intended GitHub repository, potentially exposing API keys or enabling remote code execution (running malicious commands on a system). This bug affects how the load_chain function handles file paths.

Attackers can create conditional prompt injection attacks (tricking an AI by hiding malicious instructions in its input that activate only for specific users) against Microsoft Copilot by leveraging user identity information like names and job titles that the AI includes in its context. A researcher demonstrated this by sending an email with hidden instructions that made Copilot behave differently depending on which person opened it, showing that LLM applications become more vulnerable as attackers learn to target specific users rather than all users equally.

A critical vulnerability was found in LangChain's langchain_community library version 0.0.26 in the TFIDFRetriever component (a tool that retrieves relevant documents for AI systems). The flaw allows server-side request forgery (SSRF, where an attacker tricks a server into making unwanted network requests on their behalf), and it can be exploited remotely.

The EU AI Act is a regulatory framework that requires companies to comply with rules for different types of AI systems on specific timelines, starting with prohibitions on the riskiest AI uses within 6 months and expanding to cover high-risk AI systems (such as those used in law enforcement, hiring, or education) by 24 months after the law takes effect. The article outlines key compliance deadlines, secondary laws the EU Commission might create to clarify the rules, and guidance documents to help organizations understand how to follow the AI Act.

ZenML Server in the ZenML machine learning package before version 0.46.7 has a remote privilege escalation vulnerability (CVE-2024-25723), meaning an attacker can gain higher-level access to the system from a distance. The flaw exists in a REST API endpoint (a web-based interface for requests) that activates user accounts, because it only requires a valid username and new password to change account settings, without proper access controls checking who should be allowed to do this.

The EU AI Act classifies AI systems by risk level, from prohibited (like social scoring systems that manipulate behavior) to minimal risk (unregulated). High-risk AI systems, such as those used in critical decisions affecting people's lives, face strict regulations requiring developers to provide documentation, conduct testing, and monitor for problems. General-purpose AI (large language models that can do many tasks) have lighter requirements unless they present systemic risk, in which case developers must test them against adversarial attacks (attempts to trick or break them) and report serious incidents.

CVE-2024-27444 is a vulnerability in LangChain Experimental (a Python library for building AI applications) before version 0.1.8 that allows attackers to bypass a previous security fix and run arbitrary code (malicious commands they choose) by using Python's special attributes like __import__ and __globals__, which were not blocked by the pal_chain/base.py security checks.