AI Sec Watch

Qdrant version 1.9.0-dev has a vulnerability in its snapshot recovery process (a feature that restores a database from a backup) that allows attackers to read and write arbitrary files on the server by inserting symlinks (shortcuts to other files) into snapshot files. This could potentially give attackers complete control over the system.

The Vanna library (a tool for generating data visualizations) has a vulnerability where attackers can use prompt injection (tricking an AI by hiding instructions in its input) to alter how the library processes user requests and run arbitrary Python code instead of creating the intended visualization. This happens when external input is sent to the library's 'ask' method with visualization enabled, which is the default setting, leading to remote code execution (attackers being able to run commands on a system they don't own).

Ollama versions before 0.1.34 have a security flaw where they don't properly check the format of digests (sha256 hashes that should be exactly 64 hexadecimal digits) when looking up model file paths. This allows attackers to bypass security checks by using invalid digest formats, such as ones with too few digits, too many digits, or paths starting with '../' (a path traversal technique that accesses files outside the intended directory).

A code injection vulnerability (injecting malicious code into a system) exists in the huggingface/text-generation-inference repository's workflow file, where user input from GitHub branch names is unsafely used to build commands. An attacker can exploit this by creating a malicious branch name and submitting a pull request, potentially executing arbitrary code on the GitHub Actions runner (the automated system that runs tests and builds for the project).

Qdrant version 1.9.0-dev has a path traversal vulnerability (a security flaw where an attacker manipulates file paths to access unintended locations) in its snapshot upload endpoint that allows attackers to write files anywhere on the server by encoding special characters in the request. This could lead to complete system compromise through arbitrary file upload and overwriting.

EmbedAI has a security flaw that allows data poisoning attacks (injecting false or harmful information into an AI system) through a CSRF vulnerability (cross-site request forgery, where an attacker tricks a user into performing unwanted actions on a website they're logged into). An attacker can direct users to a malicious webpage that exploits weak session management and CORS policies (which control what external websites can access the application), tricking them into uploading bad data that corrupts the application's language model.

ChatGPT's browsing tool can be tricked into automatically invoking other tools (like image creation or memory management) when users visit websites containing hidden instructions, a vulnerability known as prompt injection (tricking an AI by hiding instructions in its input). While OpenAI added some protections, minor prompting tricks can bypass them, and this issue affects other AI applications as well.

The Testimonial Carousel For Elementor WordPress plugin (versions up to 10.2.0) has a missing authorization check in the 'save_testimonials_option_callback' function, allowing unauthenticated attackers to modify data like OpenAI API keys without permission. This vulnerability is classified as CWE-862 (missing authorization, where a system doesn't verify that a user has permission to perform an action).

This overview discusses the European AI Act and the governance framework needed to implement it, focusing on the European Commission's responsibilities and the AI Office. Key tasks include establishing guidelines for classifying high-risk AI systems, defining what counts as significant modifications (changes that alter a system's risk level), and setting standards for transparency and enforcement across EU member states.