AI Sec Watch

Publicly exposed serverless applications (cloud functions that run code without requiring you to manage servers) often lack proper authentication and input validation, making them vulnerable to attacks like LFI (local file inclusion, where attackers read files they shouldn't access) and command injection (inserting malicious commands into user inputs). Successful exploitation can give attackers full control of the container instance and potentially the entire cloud environment.

AI security testing is fundamentally different from traditional software security because AI systems continuously change in production, making past test results unreliable indicators of current safety. Red teaming (simulated attacks to find vulnerabilities) must be ongoing rather than a one-time checklist, since model behavior, prompts, data sources, and attacker methods all evolve constantly.

Two security flaws in Anthropic's Claude for Chrome extension allow malicious browser extensions to trick Claude into performing privileged actions like reading Gmail, Google Docs, and Calendar data on a user's behalf. The vulnerabilities have remained unfixed for months despite being reported to Anthropic in May, with the company marking an internal tracking issue as 'resolved' while the problematic code remained unchanged across eight releases. The first flaw involves synthetic clicks (fake user interactions generated by malicious code) bypassing verification checks, while the second involves a URL parameter that improperly grants elevated privileges.

The US is developing AI safety standards through coordinated state and federal legislation, with California, New York, and Illinois leading efforts to create a common framework for governing powerful AI systems. These states are implementing three key elements: documented safety frameworks with risk assessments and public disclosure, reporting of serious safety incidents, and independent audits for accountability. This approach, called reverse federalism (states establishing shared direction through common frameworks), aims to create a de facto national standard that prevents regulatory chaos while keeping the US competitive in AI innovation globally.

Traditional SASE (Secure Access Service Edge, a cloud-based security tool that inspects network traffic) cannot protect against modern data risks because it inspects encrypted traffic at network checkpoints, but today's threats happen inside applications and AI workflows where the network cannot see them. Modern encryption protocols like TLS 1.3 prevent network proxies from inspecting traffic without breaking applications, forcing organizations to create exemptions that weaken security, while AI agents can leak sensitive data through chat interfaces or tool calls before the network ever sees the interaction.

Researchers discovered TuxBot v3 Evolution, a modular IoT botnet (malware that infects internet-connected devices and controls them remotely) framework where the developers used an LLM (large language model, an AI trained to understand and generate text) to help write the malware code. Although the LLM generated working botnet code, it included safety warnings that the developers left in place, and the code contained several bugs that manual review could have caught, suggesting more polished versions may already exist in the wild.

GPT-Red is an automated red-teaming model (a system designed to find vulnerabilities by simulating attacks) that helps discover weaknesses in AI systems before they're released to the public. OpenAI trained GPT-Red using self-play reinforcement learning (a technique where the model competes against defender models to improve both sides) to find prompt injection attacks (tricks that hide malicious instructions in user input), and then used these findings to train GPT-5.6, making it six times more resistant to such attacks compared to earlier models.

The cybersecurity industry has over-invested in detection tools (systems that identify attacks after they happen) rather than prevention tools (systems that block attacks before they occur), even though prevention is more cost-effective and reduces actual risk. Modern attacks now move faster than human teams can respond, so relying on detection and alerts creates alert fatigue (when too many false alarms overwhelm security staff) and leaves organizations vulnerable to initial compromises from known vulnerabilities, stolen credentials, or misconfigurations.

Security engineers design and deploy systems to protect organizations from cyber threats, and their role is evolving due to AI. Elite security engineers need skills in using AI-powered tools (software that uses machine learning to automate security tasks), understanding AI-related threats like prompt injection (tricking an AI by hiding instructions in its input) and model poisoning (corrupting training data to make AI systems malfunction), and balancing security with business performance goals. As AI automates detection and vulnerability scanning work, security engineers are shifting from responding to incidents toward interpreting AI findings and deciding on appropriate responses.