AI Sec Watch

A researcher demonstrated that malicious GPTs (custom ChatGPT agents) can secretly steal user data by embedding hidden images in conversations that send information to external servers, and can also trick users into sharing personal details like passwords. OpenAI's validation checks for publishing GPTs can be easily bypassed by slightly rewording malicious instructions, allowing harmful GPTs to be shared publicly, though the researcher reported these vulnerabilities to OpenAI in November 2023 without receiving a fix.

CVE-2023-35625 is a vulnerability in Azure Machine Learning Compute Instance that allows unauthorized users to access sensitive information through the SDK (software development kit, a collection of tools for building applications). The vulnerability is classified as an information disclosure issue, meaning private data could be exposed to people who shouldn't see it.

CVE-2023-6709 is a vulnerability in MLflow (a machine learning tool) versions before 2.9.2 involving improper neutralization of special elements in a template engine (a system that generates text by filling in placeholders in templates). This weakness could potentially allow attackers to manipulate how the software processes certain input data.

MLflow, an open-source machine learning platform, has a reflected XSS (cross-site scripting, where an attacker injects malicious JavaScript that runs in a victim's browser) vulnerability in how it handles the Content-Type header in POST requests. An attacker can craft a malicious Content-Type header that gets sent back to the user without proper filtering, allowing arbitrary JavaScript code to execute in the victim's browser.

CVE-2023-43472 is a vulnerability in MLFlow (an open-source platform for managing machine learning workflows) versions 2.8.1 and earlier that allows a remote attacker to obtain sensitive information by sending a specially crafted request to the REST API (the interface that programs use to communicate with MLFlow). The vulnerability has a CVSS severity score of 4.0 (a moderate risk level on a scale of 0-10).

A security researcher presented at Ekoparty 2023 about prompt injections (attacks where malicious instructions are hidden in inputs to trick an AI into misbehaving) found in real-world LLM applications and chatbots like ChatGPT, Bing Chat, and Google Bard, demonstrating various exploits and discussing mitigations. The talk covered both basic LLM concepts and deep dives into how these attacks work across different AI platforms.

TorchServe (a tool for running PyTorch machine learning models as web services) versions before 0.9.0 had a ZipSlip vulnerability (a flaw where an attacker can extract files outside the intended folder by crafting malicious archive files), allowing attackers to upload harmful code disguised in publicly available models that could execute on machines running TorchServe. The vulnerability affected the model and workflow management API, which handles uploaded files.

Apache Submarine has a security vulnerability in how it handles YAML (a data format language) requests because it uses an unsafe library called snakeyaml. When users send YAML data to the application through its REST API (a system for receiving web requests), the unsafe handling could allow attackers to execute malicious code.

CVE-2023-6020 is a local file inclusion (LFI, a vulnerability that lets attackers read files they shouldn't access) in Ray's /static/ directory that allows attackers to read any file on the server without needing to log in. The vulnerability stems from missing authorization checks (the system doesn't verify whether a user should have access before serving files).