AI Sec Watch

Prompt injection (tricking an AI by hiding instructions in its input) is a widespread vulnerability in AI education, with indirect prompt injections being particularly dangerous because they allow untrusted data to secretly take control of an LLM (large language model) and change its goals and behavior. Since attack payloads use natural language, attackers can craft many creative variations to bypass input validation (checking that data meets safety rules) and web application firewalls (security systems that filter harmful requests).

The AI ChatBot WordPress plugin before version 4.4.9 has two security flaws in its code that handles OpenAI settings. First, it lacks authorization checks (meaning it doesn't verify who should be allowed to make changes), allowing even low-privilege users like subscribers to modify settings. Second, it's vulnerable to CSRF (cross-site request forgery, where an attacker tricks a logged-in user into making unwanted changes) and stored XSS (cross-site scripting, where malicious code gets saved and runs when others view the page).

CVE-2023-2356 is a relative path traversal vulnerability (a flaw that lets attackers access files outside their intended directory by manipulating file paths) found in MLflow versions before 2.3.1. This weakness could allow attackers to read or access files they shouldn't be able to reach on systems running the affected software.

This is a podcast episode about AI red teaming (simulated attacks to find weaknesses in AI systems) and threat modeling (planning for potential security risks) in machine learning systems. The episode explores how traditional security practices can be combined with machine learning security to better protect AI applications from attacks.

IBM Watson Machine Learning on Cloud Pak for Data versions 4.0 and 4.5 has a vulnerability called SSRF (server-side request forgery, where an attacker tricks the system into making unauthorized network requests on their behalf). An authenticated attacker could exploit this to discover network details or launch other attacks.

MindsDB, a platform for building AI solutions, has a vulnerability in older versions where it unsafely extracts files from remote archives using `tarfile.extractall()` (a Python function that unpacks compressed files). An attacker could exploit this to overwrite any file that the server can access, similar to known attacks called TarSlip or ZipSlip (path traversal attacks, where files are extracted to unexpected locations).

LLM outputs are untrusted and can be manipulated through prompt injection (tricking an AI by hiding instructions in its input), which affects large language models in particular ways. This post addresses how to handle the risks of untrusted output when using AI systems in real applications.

CVE-2023-28312 is an information disclosure vulnerability in Azure Machine Learning, meaning unauthorized people could access sensitive data they shouldn't be able to see. The vulnerability involves improper access control (CWE-284, a weakness where the system doesn't properly check who is allowed to access what), and it was reported by Microsoft.

CVE-2023-29374 is a vulnerability in LangChain versions up to 0.0.131 where the LLMMathChain component is vulnerable to prompt injection attacks (tricking an AI by hiding instructions in its input), allowing attackers to execute arbitrary code through Python's exec method. This is a code execution vulnerability that could allow an attacker to run malicious commands on a system running the affected software.