AI Sec Watch

Instagram's leader Adam Mosseri warned that AI can now convincingly fake almost any content, making it hard for creators to stand out with authentic material. He proposed solving this by having camera manufacturers cryptographically sign images (using math-based codes that prove an image wasn't altered) at the moment they're captured, creating a verifiable record of what's real versus AI-generated.

Anthropic's CEO is meeting with the U.S. Defense Secretary to resolve disagreements over how the military can use the company's AI models (large language models trained to understand and generate text). Anthropic wants guarantees its technology won't be used for autonomous weapons (systems that make decisions without human control) or domestic surveillance, while the Department of Defense wants permission to use the models for any lawful purpose without restrictions.

Citrini Research published a scenario describing how AI agents (autonomous AI systems that can make decisions and take actions independently) could trigger economic collapse by replacing white-collar workers with cheaper AI alternatives, creating a negative feedback loop where job losses reduce consumer spending, forcing companies to invest even more in AI to survive. The scenario imagines unemployment doubling and stock market value falling by a third within two years, though the researchers present it as a thought experiment rather than a prediction.

The U.S. Defense Secretary is meeting with Anthropic's CEO to pressure the company into allowing military use of Claude (Anthropic's AI system) for mass surveillance and autonomous weapons (weapons that can fire without human approval). Anthropic has refused these uses, and the Pentagon is threatening to label it a "supply chain risk" (a designation that would ban it from government contracts), which could void their $200 million military contract and force other Pentagon partners to stop using Claude.

OpenAI announced partnerships with four major consulting firms (Accenture, Boston Consulting Group, Capgemini, and McKinsey) to help deploy its enterprise AI platform called Frontier, which acts as an intelligence layer that connects different systems and data within organizations to help companies manage and build AI agents (tools that can independently complete tasks). These consulting partnerships aim to accelerate AI adoption for enterprise customers by combining OpenAI's technology with the consulting firms' existing relationships and deep knowledge of how businesses operate.

This newsletter covers multiple business and policy topics, including the Supreme Court striking down Trump's tariffs (duties, or taxes on imported goods) in a 6-3 decision, followed by Trump announcing a new 15% global tariff the next day. A major winter blizzard caused airlines to cancel 15% of U.S. flights on Monday, and Trump called on Netflix to fire board member Susan Rice.

This paper presents SIMix, a training framework for systems where multiple users learn AI models together over wireless networks while protecting their private data. The system uses Over-the-Air Mixup (OAM, a technique that combines data from multiple users through wireless transmission to hide sensitive information) and groups users strategically to reduce communication needs by up to 25% while defending against model inversion attacks (attempts to reconstruct private training data from a trained model) and label inference attacks (guessing what category a user's data belongs to).

Researchers developed PPOM-Attack, a method to fool face recognition (FR) systems by generating adversarial images (slightly altered photos that trick AI into misidentifying someone). Unlike earlier attacks that used substitute models (simpler AI systems trained to mimic the target system), PPOM-Attack directly queries the real face recognition system to learn how to create effective perturbations (tiny pixel changes), achieving 21.7% higher success rates while keeping the altered images looking natural.

Prompt injection attacks (tricking an AI by hiding malicious instructions in its input) pose a serious security risk to Large Language Models, as attackers can overwrite a model's original instructions to manipulate its responses. Researchers developed PromptFuzz, a testing framework that uses fuzzing techniques (automatically generating many variations of input data to find weaknesses) to systematically evaluate how well LLMs resist these attacks. Testing showed that PromptFuzz was highly effective at finding vulnerabilities, ranking in the top 0.14% of attackers in a real competition and successfully exploiting 92% of popular LLM-integrated applications tested.