AI Sec Watch

A flaw in Microsoft 365 Copilot's permission settings allows someone with authorized access to improperly view sensitive information across a network. The issue stems from incorrect assignment of permissions (access rules) to a critical resource (important data or system component), meaning the AI tool isn't properly restricting who can see what.

Thomas Ptacek recommends using LLMs as copyediting tools rather than writing assistants, with a strict rule to never use specific phrases that an LLM suggests. He advocates for LLMs to help with fact-checking, spelling, grammar, and finding synonyms, but argues that adopting LLM-suggested wording produces text with a distinctive and undesirable quality.

vLLM (a software framework for running large language models) versions up to 0.29.0 has a memory cleanup bug in its decode workers (specialized processors that handle the generation phase of AI inference). Attackers can exploit this by sending requests with max_tokens=0 (asking for zero output tokens), which prevents the system from properly clearing temporary data, eventually consuming all available memory until the worker crashes and restarts.

M365 Copilot has a command injection vulnerability (a flaw where special characters in user input can trick the system into running unintended commands), which allows an authorized attacker to gain higher privileges over a network. The vulnerability affects users who already have some level of access to the system.

Microsoft 365 Copilot's Business Chat has a vulnerability where special characters are not properly filtered before being used in commands, allowing attackers to inject malicious commands (command injection, where an attacker sneaks unauthorized instructions into a system by exploiting how it processes input). This could let unauthorized people access and steal sensitive information across the network.

Azure Machine Learning contains a vulnerability where authorization checks (the system that verifies whether a user is allowed to perform an action) are not working correctly, allowing an attacker without permission to access and steal sensitive information over the internet.

Microsoft Copilot has a command injection vulnerability (a flaw where special characters in user input are not properly filtered, allowing attackers to execute unintended commands), which lets an unauthorized attacker access and leak sensitive information over a network.

Anthropic released three new metrics to help monitor how quickly AI is being developed, following CEO Dario Amodei's call for the AI industry to slow down its pace of advancement. The metrics measure AI-led research and development, oversight of AI agents (software that can perform tasks independently), and how computing resources are allocated within the company, with the goal of making AI development more transparent to the public so society can decide how to use this information.

AI Agent Automation, a platform for managing AI workflows with scheduling and monitoring tools, has a vulnerability in versions before 0.9.1 where authenticated users can manipulate file paths to escape the intended workspace directory and read sensitive files or overwrite files that the application can access. The vulnerability occurs because the system doesn't verify that file paths stay within approved directories after resolving them.