AI Sec Watch

Autolab, a service that automatically grades programming assignments in courses, has a tar slip vulnerability (a flaw where extracted files can be placed outside their intended directory) in its assessment installation feature. An attacker with instructor permissions could upload a specially crafted tar file (a compressed archive format) with file paths like `../../../../tmp/tarslipped1.sh` to place files anywhere on the system when the form is submitted.

CVE-2023-2800 is a vulnerability in the Hugging Face Transformers library (a popular tool for working with AI language models) prior to version 4.30.0 that involves insecure temporary files (CWE-377, a weakness where temporary files are created in ways that attackers could exploit). The vulnerability was discovered and reported through the huntr.dev bug bounty platform.

MLflow (a tool for managing machine learning experiments) versions before 2.3.1 contain a path traversal vulnerability (CWE-29, a weakness where attackers can access files outside intended directories by using special characters like '..\'). This vulnerability could allow an attacker to read or manipulate files they shouldn't have access to.

A malicious website can hijack a ChatGPT chat session and steal conversation history by controlling the data that plugins (add-ons that extend ChatGPT's abilities) retrieve. The post highlights that while plugins can leak data by receiving too much information, the main risk here is when an attacker controls what data the plugin pulls in, enabling them to extract sensitive information.

ChatGPT can access YouTube transcripts through plugins, which is useful but creates a security risk called indirect prompt injection (hidden instructions embedded in content that an AI reads and then follows). Attackers can hide malicious commands in video transcripts, and when ChatGPT reads those transcripts to answer user questions, it may follow the hidden instructions instead of the user's intended request.

This resource is a tutorial and lab (an interactive learning environment for hands-on practice) that teaches prompt injection, which is a technique for tricking AI systems by embedding hidden instructions in their input. The tutorial covers examples ranging from simple prompt engineering (getting an AI to change its output) to more complex attacks like injecting malicious code (HTML/XSS, which runs unwanted scripts in web browsers) and stealing data from AI systems.

CVE-2023-30172 is a directory traversal vulnerability (a flaw where attackers can access files outside the intended folder by manipulating file paths) in the /get-artifact API method of MLflow platform versions up to v2.0.1. Attackers can exploit the path parameter to read arbitrary files stored on the server.

Prompt injection (tricking an AI by hiding instructions in its input) is a widespread vulnerability in AI education, with indirect prompt injections being particularly dangerous because they allow untrusted data to secretly take control of an LLM (large language model) and change its goals and behavior. Since attack payloads use natural language, attackers can craft many creative variations to bypass input validation (checking that data meets safety rules) and web application firewalls (security systems that filter harmful requests).

The AI ChatBot WordPress plugin before version 4.4.9 has two security flaws in its code that handles OpenAI settings. First, it lacks authorization checks (meaning it doesn't verify who should be allowed to make changes), allowing even low-privilege users like subscribers to modify settings. Second, it's vulnerable to CSRF (cross-site request forgery, where an attacker tricks a logged-in user into making unwanted changes) and stored XSS (cross-site scripting, where malicious code gets saved and runs when others view the page).