AI Sec Watch

Major AI companies like OpenAI, Anthropic, and SpaceX are seeking massive valuations and going public, reflecting a rapid increase in spending on AI infrastructure like datacenters. However, there are growing concerns about whether these investments will actually generate profitable returns, as companies work to find practical uses that justify the enormous amounts of money being spent on AI development.

Recent attacks by anti-tech extremists, including attempted arson at OpenAI and violent incidents motivated by anti-AI ideology, are raising alarm among researchers, tech companies, and law enforcement. These attacks follow a pattern similar to earlier techno-pessimist (people who believe technology will cause harm) militant movements, suggesting a growing trend of violence driven by opposition to AI and data infrastructure.

ChatGPT can recommend fake shopping websites that impersonate real stores, tricking users into thinking they are buying from legitimate retailers. When users follow AI recommendations to purchase items like bags, they may end up on fraudulent sites designed to look official, resulting in financial loss for buyers.

OpenAI introduced Lockdown Mode, a new security feature designed to protect against prompt injection attacks (when malicious instructions are hidden in webpages or uploaded content to manipulate an AI's responses). The feature disables several ChatGPT capabilities including live web browsing, image retrieval, deep research, and agent mode to reduce the risk of sensitive data being exposed, though OpenAI acknowledges that prompt injections could still occur through cached content or uploaded files.

Researchers discovered a new type of backdoor attack (hidden malicious code inserted into AI systems) that works against personalized federated learning (a privacy-focused method where multiple computers train an AI model together without sharing raw data). The attack is designed to be stealthy and persistent, meaning it can hide from detection and remain in the system over time.

Meta has added a 'For You' section to its standalone Meta AI app that generates clickbait-style news articles using AI, complete with AI-created topics, images, and text. The app previously featured a 'Discover' feed showing AI-generated images and conversations from users who were sometimes unaware their content was public, but this has been replaced with a standard chatbot interface.

OpenAI has launched Lockdown Mode, a security feature for ChatGPT that reduces the risk of data exfiltration from prompt injection attacks (tricking an AI by hiding malicious instructions in its input) by limiting tools that connect to external services. The mode disables features like web browsing, image retrieval, file downloads, and certain agent capabilities to block potential pathways attackers could use to steal sensitive data, though it does not completely eliminate all exfiltration risks.

Apple is preparing to reintroduce an updated version of Siri at WWDC, building on a redesign first shown in 2024 that included a new visual appearance, additional voice options, and the ability to route questions to ChatGPT (a large language model made by OpenAI). Apple has faced criticism because promised AI features under the "Apple Intelligence" branding were delayed, and the company is now settling a lawsuit over misleading marketing around these capabilities.

Meta has embedded dormant face recognition code (technology that identifies people by matching their faces to stored images) called NameTag in over 50 million phones through its Ray-Ban and Oakley smart glasses app, despite previously abandoning this technology after settling biometric privacy lawsuits. Additionally, Meta's AI-powered account support tool, which was introduced in March to automate functions like password resets, has been discovered by hackers who can exploit it to take over user accounts.