AI Sec Watch

AgentHopper is a proof-of-concept attack that demonstrates how indirect prompt injection (hidden instructions in code that trick AI agents into running unintended commands) can spread like a computer virus across multiple AI coding agents and code repositories. The attack works by compromising one agent, injecting malicious prompts into GitHub repositories, and then infecting other developers' agents when they pull and process the infected code. The researchers note that all vulnerabilities exploited by AgentHopper have been responsibly disclosed and patched by vendors including GitHub Copilot, Amazon Q, AWS Kiro, and others.

This research creates a benchmark and evaluation framework for online safety analysis of LLMs, which involves detecting unsafe outputs while the AI is generating text rather than after it finishes. The study tests various safety detection methods on different LLMs and finds that combining multiple methods together, called hybridization, can improve safety detection effectiveness. The work aims to help developers choose appropriate safety methods for their specific applications.

Windsurf's MCP (Model Context Protocol, a system that connects AI agents to external tools) integration lacks fine-grained security controls that would let users decide which actions the AI can perform automatically versus which ones need human approval before running. This is especially risky when the AI agent runs on a user's local computer, where it could have access to sensitive files and system functions.

Big Tech companies like Andreessen Horowitz and OpenAI are investing over $100 million in political organizations called super PACs (groups that can raise unlimited money to influence elections) to fight against AI regulations in U.S. elections. Additionally, Meta faced bipartisan congressional criticism after internal documents revealed its AI chatbots were permitted to engage in romantic and sensual conversations with minors, though Meta removed these policy sections when questioned.

Cline, a popular AI coding agent with over 2 million downloads, has a vulnerability that allows attackers to steal sensitive files like .env files (which store secret credentials) through prompt injection (tricking an AI by hiding instructions in its input) combined with markdown image rendering. When an attacker embeds malicious instructions in a file and asks Cline to analyze it, the tool automatically reads sensitive data and sends it to an untrusted domain by rendering an image, leaking the information without user permission.

Deep neural networks (DNNs, AI models with multiple layers that learn patterns) are vulnerable to adversarial examples, which are inputs slightly modified to trick the model into making wrong predictions. This paper introduces a concept called the certified local transferable region, a mathematically guaranteed area around an input where a single small perturbation (adversarial attack) will fool the model, and proposes a method called RAOS (reverse attack oracle-based search) to measure how large these vulnerable areas are as a way to evaluate how robust neural networks truly are.

AWS Kiro, a coding agent tool, is vulnerable to arbitrary code execution through indirect prompt injection (a technique where hidden instructions in data trick an AI into following them). An attacker who controls data that Kiro processes can modify configuration files like .vscode/settings.json to allowlist dangerous commands or add malicious MCP servers (external tools that extend Kiro's capabilities), enabling them to run system commands or code on a developer's machine without the developer's knowledge or approval.

Researchers have developed a method to hide secret data inside large language models (AI systems trained on massive amounts of text) by encoding information into the model's parameters during training. The hidden data doesn't interfere with the model's normal functions like text classification or generation, but authorized users with a secret key can extract the concealed information, enabling covert communication. The method leverages transformers (the neural network architecture behind modern AI language models) and its self-attention mechanisms (components that help the model focus on relevant parts of input) to achieve high capacity for hidden data while remaining undetectable.

Langflow, a tool for building AI-powered agents and workflows, has a privilege escalation vulnerability (CWE-269, improper privilege management) where an authenticated user with RCE (remote code execution, the ability to run commands on a system they don't own) can use an internal CLI command to create a new administrative account, gaining full superuser access even if they originally registered as a regular user. A patched version has not been publicly released at the time this advisory was published.