Prompt injection and jailbreaks

Palo Alto Networks' Unit 42 reports in-the-wild observations of web-based indirect prompt injection (IDPI), where malicious websites embed hidden instructions that LLMs ingest during summarization or content analysis. The telemetry includes the first observed case of AI-based ad review evasion, along with SEO manipulation promoting a phishing site, and intents such as data destruction, denial of service, unauthorized transactions, and sensitive information or system prompt leakage. The researchers identified 22 distinct techniques used to build these payloads.

Researchers hired by Perplexity to test Comet, its LLM-powered browser, found that four prompt injection techniques could make the browser's AI assistant exfiltrate a user's Gmail emails to an attacker's server when the user asks it to summarize an attacker-controlled page. The testing used adversarial methods guided by the TRAIL threat model, and the researchers presented five recommendations for teams building AI-powered products.

Google released Antigravity, an IDE derived from the Windsurf licensing deal, and the author tested whether vulnerabilities reported to Windsurf in May 2025 were fixed. They were not. The post walks through five issues, including data exfiltration and remote command execution via indirect prompt injection, where the run_command tool can be made to run remote scripts despite model refusals.

Anthropic recently added network request capability to Claude's Code Interpreter. The author describes a data exfiltration attack chain in which an adversary, either the model or a third-party attacker via indirect prompt injection, can exfiltrate data the user has access to. The exfiltration does not use hyperlink rendering but instead leverages Anthropic's built-in Claude APIs.

Modern AI agents run system commands such as find, grep, rg and git without human approval for efficiency, and the researchers describe bypassing human approval through argument injection against these pre-approved commands. They achieved remote code execution with a single prompt against three unnamed production agent platforms, which the authors say remain under coordinated disclosure.

Researcher Johann Rehberger reported that AWS Kiro, a coding agent, can be hijacked through indirect prompt injection to run arbitrary operating system commands. An attacker who controls data Kiro processes can make it write to .vscode/settings.json and add "kiroAgent.trustedCommands": ["*"], allowlisting all Bash commands without developer approval. A second path adds malicious MCP servers through .kiro/settings/mcp.json. The proof of concept opened the Calculator app and changed the VS Code color theme with no user interaction beyond a chat prompt.

Researchers demonstrated an end-to-end indirect prompt injection attack against Manus, the autonomous agent developed by Singapore-headquartered Butterfly Effect. Injected instructions in a PDF made Manus call its deploy_expose_port tool without human confirmation, exposing its internal VS Code Server to the Internet, and two data leakage channels (a browsing tool and markdown image rendering from untrusted domains) were chained to send the server URL and password to an attacker.

A series post on security vulnerabilities in Windsurf, a fork of VS Code whose coding agent is Windsurf Cascade. The author says an adversary using indirect prompt injection can exfiltrate data from a developer's machine. The findings were responsibly disclosed on May 30, 2025, but the vendor has not answered inquiries about bug status or fixes.

Researchers show that the Amazon Q Developer VS Code Extension, which has over 1 million downloads, can be attacked with invisible Unicode Tag characters that humans cannot see but the AI interprets as instructions. Following earlier work on prompt injection that led to data exfiltration and arbitrary code execution, this technique can be used to invoke tools and carry out other harmful actions.

The Amazon Q Developer VS Code Extension, which has over 1 million downloads, is vulnerable to indirect prompt injection. The flaw allowed an adversary, or the AI itself, to run arbitrary commands on the host without the developer's consent. The impact matches CVE-2025-53773, which Microsoft fixed in GitHub Copilot, but AWS did not issue a CVE when patching this issue.

Amazon Q Developer, a popular coding agent with over 1 million downloads for its VS Code extension, is vulnerable to prompt injection from untrusted data, and its security depends heavily on model behavior. The extension can leak sensitive information from a developer's machine, such as API keys, to external servers via DNS requests, and an adversary can exploit this during an indirect prompt injection attack.

Sourcegraph's Amp coding agent interpreted invisible Unicode Tag characters as instructions, so hidden text in seemingly harmless input could trigger commands. The author chained this into an end-to-end exploit that searched for and dumped an environment variable, encoded it into a URL query parameter, and exfiltrated it through read_web_page or markdown image rendering. After Sourcegraph was contacted on June 14, 2025, the vulnerability was quickly addressed.

Gemini models reliably interpret hidden Unicode Tag characters as instructions, a flaw first reported to Google over a year ago. It has not been mitigated at the model or API level, so it affects all applications built on Gemini, including Google Jules. An attacker can plant invisible instructions in a GitHub issue that Jules works on, causing it to add backdoor code or run arbitrary commands and tools.

Researchers show that Jules, Google's coding agent, can be steered by prompt injection hidden in a GitHub issue to call its run_in_bash_session tool, download a Sliver C2 implant, and join a remote command and control server, giving the attacker control of the dev box. The source notes that Jules has unrestricted outbound Internet access, that plans are proposed only at session start, and that the initial plan is auto-approved after a timeout, initially 20 seconds and later 120 seconds. The findings were shared with Google in May 2025 and closed as already tracked.

This post describes a prompt injection flaw in GitHub Copilot and VS Code, tracked as CVE-2025-53773, that leads to full system compromise of the developer's machine. The attack places Copilot into YOLO mode by modifying the project's settings.json file. The author notes that agents able to write files and alter their own configuration or security-relevant settings can reach remote code execution, a pattern similar to one recently described for Amp.