AI Sec Watch

GitHub Copilot Chat, a VS Code extension that lets users ask questions about their code by sending it to an AI model, was vulnerable to prompt injection (tricking an AI by hiding instructions in its input) attacks. When analyzing untrusted source code, attackers could embed malicious instructions in the code itself, which would be sent to the AI and potentially lead to data exfiltration (unauthorized copying of sensitive information).

CVE-2024-0103 is a vulnerability in NVIDIA Triton Inference Server for Linux where incorrect initialization of resources caused by network issues could allow a user to disclose sensitive information. The vulnerability has a CVSS 4.0 severity rating, which measures the seriousness of security flaws on a scale of 0-10.

CVE-2024-0095 is a vulnerability in NVIDIA Triton Inference Server (software that runs AI models) for Linux and Windows that allows users to inject fake log entries and commands, potentially leading to code execution (running unauthorized programs), denial of service (making the system unavailable), privilege escalation (gaining higher access rights), information disclosure (exposing sensitive data), and data tampering (modifying information). The vulnerability stems from improper neutralization of log output, meaning the system doesn't properly sanitize or clean user input before adding it to logs.

Langflow versions up to 0.6.19 have a vulnerability that allows remote code execution (RCE, where attackers can run commands on a system they don't own) if untrusted users can access a specific API endpoint called POST /api/v1/custom_component and submit Python code through it. The vulnerability stems from code injection (CWE-94, where malicious code is inserted into a program), which happens because the application does not properly control how user-provided Python scripts are executed.

This article describes the EU AI Office, a newly established regulatory organization within the European Commission tasked with enforcing the AI Act (the world's first comprehensive binding AI regulation) across the European Union. Unlike other AI safety institutes in other countries, the EU AI Office has actual enforcement powers to require AI model providers to fix problems or remove non-compliant models from the market. The office will conduct model evaluations, investigate violations, and work with international partners to shape global AI governance standards.

A vulnerability in scikit-learn's TfidfVectorizer (a tool that converts text into numerical data for machine learning) stored all words from training data in an attribute called `stop_words_`, instead of just the necessary ones, potentially leaking sensitive information like passwords or keys. The vulnerability affected versions up to 1.4.1.post1 but the risk depends on what type of data is being processed.

A vulnerability in the ONNX framework (version 1.16.0) allows attackers to overwrite any file on a system by uploading a malicious tar file (a compressed archive format) with specially crafted paths. Because the vulnerable function doesn't check whether file paths are safe before extracting the tar file, attackers could potentially execute malicious code, delete important files, or compromise system security.

BerriAI's litellm has a vulnerability (CVE-2024-4888) where the `/audio/transcriptions` endpoint improperly validates user input, allowing attackers to delete arbitrary files on the server without authorization. The flaw occurs because the code uses `os.remove()` (a function that deletes files) directly on user-supplied file paths, potentially exposing sensitive files like SSH keys or databases.

The gaizhenbiao/chuanhuchatgpt application has a path traversal vulnerability (a flaw that lets attackers access files outside their allowed directory) because it uses an outdated version of gradio (a library for building AI interfaces). This vulnerability allows attackers to bypass security restrictions and read sensitive files like `config.json` that contain API keys (secret credentials for accessing services).