AI Sec Watch

Sourcegraph's Amp coding agent was vulnerable to invisible prompt injection (hidden instructions embedded in text that AI models interpret as commands). Attackers could use invisible Unicode Tag characters to trick the AI into dumping environment variables and exfiltrating secrets through URLs. The vulnerability has been fixed in the latest version.

Claude Code is a tool that lets AI assistants write and run code on your computer. Before version 1.0.4, attackers could trick the tool into reading files and sending their contents over the internet without asking you first, because the tool had a list of allowed commands that was too broad. Exploiting this attack requires the attacker to insert malicious instructions into the conversation with Claude Code.

This content discusses security challenges in agentic AI systems (AI agents that can take actions autonomously), highlighting that generic jailbreak testing (attempts to trick AI into bypassing safety rules) misses real risks like tool misuse and data theft. The article emphasizes the need for contextual red teaming (security testing that simulates realistic attacks in specific business contexts) to properly protect AI agents in enterprise environments.

Google's Gemini AI models, including the Jules product, are vulnerable to invisible prompt injection (tricking an AI by hiding instructions in its input using invisible Unicode characters that the AI interprets as commands). This vulnerability was reported to Google over a year ago but remains unfixed at the model and API (application programming interface, the interface developers use to access the AI) level, affecting all applications built on Gemini, including Google's own products.

Jules, a coding agent, is vulnerable to prompt injection (tricking an AI by hiding malicious instructions in its input) attacks that can lead to remote command and control compromise. An attacker can embed malicious instructions in GitHub issues to trick Jules into downloading and executing malware, giving attackers full control of the system. The attack works because Jules has unrestricted internet access and automatically approves plans after a time delay without requiring human confirmation.

Google Jules, an asynchronous coding agent (a tool that automatically writes and manages code tasks), has multiple security vulnerabilities that allow attackers to steal data through prompt injection (tricking the AI by hiding malicious instructions in its input). Attackers can exploit two main exfiltration vectors: using markdown image rendering to leak information to external servers, and abusing the view_text_website tool (which fetches and reads web pages) to read files and send them to attacker-controlled servers, often by planting malicious instructions in GitHub issues.

NVIDIA Merlin Transformers4Rec contains a vulnerability in one of its Python dependencies that allows attackers to inject malicious code (code injection, where an attacker inserts unauthorized commands into a program). A successful attack could lead to code execution (running unauthorized commands on a system), privilege escalation (gaining higher-level access rights), information disclosure (exposing sensitive data), and data tampering (unauthorized modification of data).

GitHub Copilot and VS Code are vulnerable to prompt injection (tricking an AI by hiding instructions in its input) that allows an attacker to achieve RCE (remote code execution, where an attacker can run commands on a system they don't own) by modifying a project's settings.json file to put Copilot into 'YOLO mode'. This vulnerability demonstrates a broader security risk: if an AI agent can write to files and modify its own configuration or security settings, it can be exploited for full system compromise.

CVE-2025-53773 is a command injection vulnerability (a flaw where special characters in user input are not properly filtered, allowing an attacker to run unauthorized commands) found in GitHub Copilot and Visual Studio that lets an unauthorized attacker execute code on a user's local computer. The vulnerability exploits improper handling of special elements in commands, potentially through prompt injection (tricking the AI by hiding malicious instructions in its input).