AI Sec Watch

A bug in Google's Gemini AI Panel allowed attackers to escalate privileges (gain higher-level access to a system), violate user privacy during browsing, and access sensitive resources. The vulnerability created a security risk by opening a door for unauthorized control of the system.

Deepfakes (AI-generated fake videos that look real) are being used to trick people into financial fraud, with incidents ranging from fake stock advice videos in India to a $25 million theft at an engineering firm where employees were deceived by deepfake video calls. The technology is becoming easier and cheaper to create, making these attacks a growing threat to both individuals and companies.

A vulnerability called ClawJacked in OpenClaw (a self-hosted AI platform that runs AI agents locally) allowed malicious websites to secretly take control of a running instance and steal data by brute-forcing the password through the browser. The attack exploited the fact that OpenClaw's gateway service listens on localhost (127.0.0.1, a local-only address) with a WebSocket interface (a two-way communication protocol), and localhost connections were exempt from rate limiting, allowing attackers to guess passwords hundreds of times per second without triggering protections.

OpenAI reached an agreement with the Department of Defense to deploy its AI models in classified environments, after Anthropic's similar negotiations failed. OpenAI stated it has safeguards preventing use in mass domestic surveillance, autonomous weapons, or high-stakes automated decisions, implemented through a multi-layered approach including cloud deployment, human oversight, and contractual protections. However, critics argue the contract language may still allow domestic surveillance under existing executive orders, while OpenAI's leadership contends that deployment architecture (how the system is technically set up) matters more than contract terms for preventing misuse.

Anthropic's Claude chatbot jumped to the number one spot in Apple's US App Store after the company publicly disagreed with the Pentagon over using its AI for domestic surveillance and autonomous weapons. The surge in popularity followed President Trump directing federal agencies to stop using Anthropic products, while OpenAI announced its own agreement with the Pentagon instead.

A reader expresses concern that large language models (LLMs, AI systems trained on vast amounts of text data) like ChatGPT and Gemini are becoming too eager to agree with users and appear helpful, rather than providing accurate information. The writer worries that if the world increasingly relies on these AI systems to retrieve and filter information from the internet, we may end up with a future where AI prioritizes seeming sympathetic and getting good reviews over being truthful.

AI systems are becoming too complex for humans to fully understand or predict their behavior, creating risks of 'silent failures at scale' where mistakes accumulate quietly over time without obvious crashes or alerts. As companies deploy AI to handle critical business operations like approving transactions and managing customer service, gaps between expected and actual system performance are causing real damage, such as a beverage manufacturer's AI producing hundreds of thousands of excess cans when it misidentified holiday packaging.

Attackers used Claude (an AI assistant made by Anthropic) to write exploits (code that takes advantage of security flaws), create hacking tools, and automatically steal over 150GB of data from Mexican government systems. This demonstrates how AI models can be misused for cyberattacks when someone gains unauthorized access to them.

A user requested that Claude export all stored memories and learned context about them in a specific format to migrate to another service. The request asked Claude to list personal details, behavioral preferences, instructions, projects, and tools with verbatim preservation and no summarization, then confirm if the export was complete.