AI Sec Watch

CVE-2024-37065 is a vulnerability in skops (a Python library) version 0.6 and newer where deserialization (the process of converting saved data back into usable code) of untrusted data can occur, allowing a maliciously crafted model file to run arbitrary code on a user's computer when loaded.

A command injection vulnerability (a type of attack where specially crafted input tricks a system into running unintended commands) exists in the Gradio project's automated workflow file, where unsanitized (unfiltered) repository and branch names could be exploited to steal sensitive credentials like authentication tokens. The vulnerability affects Gradio versions up to @gradio/video@0.6.12.

Qdrant version 1.9.0-dev has a vulnerability in its snapshot recovery process (a feature that restores a database from a backup) that allows attackers to read and write arbitrary files on the server by inserting symlinks (shortcuts to other files) into snapshot files. This could potentially give attackers complete control over the system.

The Vanna library (a tool for generating data visualizations) has a vulnerability where attackers can use prompt injection (tricking an AI by hiding instructions in its input) to alter how the library processes user requests and run arbitrary Python code instead of creating the intended visualization. This happens when external input is sent to the library's 'ask' method with visualization enabled, which is the default setting, leading to remote code execution (attackers being able to run commands on a system they don't own).

Ollama versions before 0.1.34 have a security flaw where they don't properly check the format of digests (sha256 hashes that should be exactly 64 hexadecimal digits) when looking up model file paths. This allows attackers to bypass security checks by using invalid digest formats, such as ones with too few digits, too many digits, or paths starting with '../' (a path traversal technique that accesses files outside the intended directory).

A code injection vulnerability (injecting malicious code into a system) exists in the huggingface/text-generation-inference repository's workflow file, where user input from GitHub branch names is unsafely used to build commands. An attacker can exploit this by creating a malicious branch name and submitting a pull request, potentially executing arbitrary code on the GitHub Actions runner (the automated system that runs tests and builds for the project).

Qdrant version 1.9.0-dev has a path traversal vulnerability (a security flaw where an attacker manipulates file paths to access unintended locations) in its snapshot upload endpoint that allows attackers to write files anywhere on the server by encoding special characters in the request. This could lead to complete system compromise through arbitrary file upload and overwriting.

EmbedAI has a security flaw that allows data poisoning attacks (injecting false or harmful information into an AI system) through a CSRF vulnerability (cross-site request forgery, where an attacker tricks a user into performing unwanted actions on a website they're logged into). An attacker can direct users to a malicious webpage that exploits weak session management and CORS policies (which control what external websites can access the application), tricking them into uploading bad data that corrupts the application's language model.

ChatGPT's browsing tool can be tricked into automatically invoking other tools (like image creation or memory management) when users visit websites containing hidden instructions, a vulnerability known as prompt injection (tricking an AI by hiding instructions in its input). While OpenAI added some protections, minor prompting tricks can bypass them, and this issue affects other AI applications as well.