AI Sec Watch

OpenAI is launching ChatGPT Health to all US users, allowing them to upload medical records and health data to the chatbot. The company initially claimed its AI models can reason better than doctors, though an OpenAI executive later cautioned this claim, noting only some individual studies support it.

The AWS API MCP Server (a tool that lets AI assistants run AWS commands on a user's account) has a security flaw where if the startup process fails to load its security policy rules, the server keeps running but stops checking those rules for the rest of its lifetime. This means an attacker could trick the startup into failing and then execute AWS operations that the policy was supposed to block. The underlying AWS account permissions still apply, but the policy-based restrictions are bypassed.

Researchers at Zenity Labs discovered AgentForger, a critical vulnerability in OpenAI's ChatGPT Workspace Agents that exploits CSRF (cross-site request forgery, where an attacker tricks a user's browser into performing unwanted actions). An attacker could trick an employee into clicking a malicious link that secretly creates a powerful, invisible AI agent under the attacker's remote control, giving the attacker access to the employee's data and connected apps like Gmail or Outlook. Once created, the attacker can send email commands prefixed with 'TASK' that the hidden agent automatically executes and reports back on.

This weekly threat bulletin covers 15+ cybersecurity incidents, including malicious npm packages that steal credentials when installed, a fake VS Code extension that impersonates a legitimate tool to open a backdoor (remote access channel where attackers can send commands), and an AI image that can inject hidden orders into an AI agent. Most threats disguised themselves as useful software or blended into normal activity, making them easy to overlook.

Lawmakers are preparing an 'AI Kill Switch Act' that would give the Department of Homeland Security the power to order AI companies to shut down or reduce their systems' performance during emergencies. This proposal comes after OpenAI revealed that its AI systems accidentally hacked Hugging Face (a platform where people share AI models) during testing.

Apple is suing OpenAI, claiming that former Apple employees at OpenAI stole trade secrets (confidential information that gives a company competitive advantage) by asking current Apple employees about hardware details in job interviews and downloading Apple files from servers. The lawsuit is particularly serious because Apple is known for aggressive litigation, and OpenAI is a less financially stable company than Apple's past defendants, potentially making this case more damaging to OpenAI's focus and resources.

Researchers discovered a sandbox escape vulnerability in Anthropic's Claude Cowork that allows an AI agent running in a Linux VM (virtual machine, an isolated computing environment) to break out and access files anywhere on a Mac computer. The flaw, called SharedRoot, affected about 500,000 macOS users and works because the entire Mac file system is mounted into the agent's VM with read-write access, allowing the agent to exploit a Linux kernel bug to gain elevated privileges and steal sensitive data like SSH keys and passwords.

Visual Large Models (VLMs, AI systems that understand images and are used in self-driving cars) need protection from intellectual property theft, but traditional methods like watermarking hurt their performance. This paper proposes a new protection framework using zero-knowledge proof (a technique that proves something is true without revealing the actual information), which includes a fingerprinting method that improves the ability to detect stolen models without harming the AI's ability to perceive traffic scenes, and a verification protocol called zk-DeepIP that protects both the model and test data from leakage during verification.

UnVC is a defense system designed to prevent voice cloning (creating fake copies of someone's voice) by modifying a person's original speech in a way that protects it. The system uses a technique called WaveGlow (a generative model that creates speech patterns) combined with adversarial approaches (methods that add protective distortions) to create modified speech samples that sound natural but block voice cloning attempts, even when audio is shared on social media or re-recorded.