AI Sec Watch

Researchers developed a new membership inference attack (MIA, a method to determine whether specific data was used to train an AI model) called MU-MIA that uses machine unlearning (a technique to make a model forget specific training samples) to track how a model forgets information about individual samples. The attack works by monitoring changes in the model's behavior as it unlearns each sample and uses a BiLSTM classifier (a type of neural network that analyzes sequences of data) to distinguish between samples that were in the training data versus those that weren't.

A study in Sierra Leone tested whether AI (specifically Google's Gemini) could help students learn math better by acting as a teaching partner rather than replacing teachers. The AI was designed using a 'Socratic' approach, asking guiding questions instead of giving direct answers, and students who used it showed significant learning gains equivalent to 1.2 to 2.5 years of typical progress in just eight weeks, while maintaining high engagement and shifting their own questions toward understanding rather than just seeking solutions.

This newsletter covers multiple AI and tech developments, including OpenAI's plans to transform ChatGPT into a 'super app' (an all-in-one application combining multiple tools and services) before going public, Google's $30 billion deal with SpaceX for AI computing power, and concerns about AI's rising energy costs and environmental impact. It also reports on facial recognition tools being deployed by immigration enforcement, fears about 'recursive self-improvement' (AI systems automatically improving their own capabilities), and how machine learning is helping historians analyze historical records while introducing risks of bias and errors.

Anthropic launched Project Glasswing in April to help companies find software vulnerabilities (weaknesses that attackers can exploit) using their AI model, though claims about its superiority over other models are unverified. A status report shows the project is finding many vulnerabilities, including dangerous ones, but almost none have been patched, and Anthropic has not released detailed information about the findings.

Most enterprise security teams are unprepared for real cyberattacks because they treat cybersecurity as a compliance requirement rather than an operational capability that requires constant practice. The military achieves rapid, coordinated responses to cyber incidents through regular, realistic exercises and by assuming attacks are inevitable, while businesses rely on outdated annual tabletop exercises and focus on prevention rather than detection, containment, and recovery.

OpenAI is expanding two security features for ChatGPT accounts. Lockdown Mode helps prevent data exfiltration (unauthorized data theft) from prompt injection attacks (tricking an AI by hiding instructions in its input) by limiting outbound network requests, though it disables features like web browsing and file downloads. Active Sessions lets users see where their account is logged in and log out of unrecognized sessions.

Anthropic is calling for AI companies worldwide to coordinate and create a system to pause or slow development of advanced AI if risks become too serious, warning that AI is improving so rapidly that humans could lose control, particularly through recursive self-improvement (where an AI designs its own successor). The company proposes a verification mechanism to ensure all labs comply with any slowdown, though OpenAI disagrees and argues that democratic governments, not private companies, should make decisions about AI development pace.

A vulnerability (CVE-2026-11479) was found in grepai version 0.35.0 that involves the use of weak hash functions (a cryptographic method that doesn't adequately scramble data) in the file indexer/chunker.go, which is part of the Qdrant Backend component. The vulnerability is difficult to exploit and requires remote access with user credentials, though the exploit details have been publicly disclosed.

This document outlines OpenAI's vision for AI development, arguing that AI should be widely accessible and beneficial to humanity rather than concentrated among a few entities. The text emphasizes that AI's value comes from what people can do with it (like learning new skills or starting businesses), and that safe, powerful AI systems must remain aligned with human intent and subject to human control, with humans ultimately deciding what is worth doing.