AI Sec Watch

A security researcher presented at Ekoparty 2023 about prompt injections (attacks where malicious instructions are hidden in inputs to trick an AI into misbehaving) found in real-world LLM applications and chatbots like ChatGPT, Bing Chat, and Google Bard, demonstrating various exploits and discussing mitigations. The talk covered both basic LLM concepts and deep dives into how these attacks work across different AI platforms.

TorchServe (a tool for running PyTorch machine learning models as web services) versions before 0.9.0 had a ZipSlip vulnerability (a flaw where an attacker can extract files outside the intended folder by crafting malicious archive files), allowing attackers to upload harmful code disguised in publicly available models that could execute on machines running TorchServe. The vulnerability affected the model and workflow management API, which handles uploaded files.

Apache Submarine has a security vulnerability in how it handles YAML (a data format language) requests because it uses an unsafe library called snakeyaml. When users send YAML data to the application through its REST API (a system for receiving web requests), the unsafe handling could allow attackers to execute malicious code.

CVE-2023-6020 is a local file inclusion (LFI, a vulnerability that lets attackers read files they shouldn't access) in Ray's /static/ directory that allows attackers to read any file on the server without needing to log in. The vulnerability stems from missing authorization checks (the system doesn't verify whether a user should have access before serving files).

CVE-2023-6014 is a vulnerability in MLflow (a machine learning experiment tracking platform) that allows attackers to create user accounts without proper authentication (the process of verifying someone's identity). The vulnerability has a CVSS score (a 0-10 rating of how severe a vulnerability is) of 4.0, indicating moderate severity.

CVE-2023-6021 is a local file inclusion (LFI, a vulnerability where an attacker can read files from a server by manipulating file paths) in Ray's log API endpoint that allows attackers to read any file on the server without needing authentication. The vulnerability affects Ray versions before 2.8.1.

CVE-2023-6018 is a vulnerability in MLflow (an open-source machine learning platform) that allows an attacker to overwrite any file on the server without needing to log in or authenticate. The vulnerability is caused by OS command injection (a flaw where special characters in user input are not properly filtered before being executed as system commands), which gives attackers the ability to run unauthorized commands on the server.

CVE-2023-6015 is a vulnerability in MLflow that allows attackers to upload arbitrary files to the server using PUT requests. This is a path traversal vulnerability (CWE-22, where an attacker can write files outside the intended directory by manipulating file paths), with a CVSS severity score of 4.0 (a moderate-level security issue on a 0-10 scale).

CVE-2023-5245 is a vulnerability in FileUtil.extract() where zip file extraction does not check if file paths are outside the intended directory, allowing attackers to create files anywhere and potentially execute code when TensorflowModel processes a saved model. This is called path traversal (a technique where an attacker uses file paths like '../../../' to escape a restricted folder).