AI Sec Watch

OpenAI is acquiring Promptfoo, a cybersecurity startup that provides tools to test and secure AI systems, particularly as AI agents (autonomous programs that can take actions) become more connected to real data and systems. Promptfoo's security tools will be integrated into OpenAI's Frontier platform, and OpenAI will continue supporting Promptfoo's open-source project that helps developers test different AI prompts and compare large language models (AI systems trained on massive amounts of text data).

OpenAI acquired Promptfoo, an AI security startup, to integrate its technology into OpenAI's enterprise platform for protecting AI agents from attacks. Promptfoo develops tools that help companies test security vulnerabilities in LLMs (large language models, the AI systems behind chatbots), addressing growing concerns that autonomous AI agents could be exploited to steal data or manipulate systems.

Anthropic, a major AI company, is suing the US Department of Defense after being labeled a supply-chain risk (a company whose products or services might pose security threats if compromised). The lawsuit claims the Trump administration retaliated against Anthropic for refusing to remove safety restrictions on its AI systems, particularly regarding mass surveillance and fully autonomous weapons (systems that make lethal decisions without human involvement).

Anthropic, an AI company, is suing the US Department of Defense after being labeled a 'supply chain risk' (a designation meaning the government considers the company a potential threat to national security in government contracts). The lawsuit claims this blacklisting is unlawful and violates free speech rights, stemming from a dispute over Anthropic's safety measures designed to prevent the military from using its AI models for mass surveillance or fully autonomous weapons.

Anthropic, an AI company, sued the Trump administration after being blacklisted and designated a supply chain risk (a classification usually reserved for foreign threats), which prevents the Pentagon and its contractors from using the company's AI models. The lawsuit claims the blacklist is unlawful and is causing irreparable harm by canceling government contracts and jeopardizing hundreds of millions of dollars in business. The conflict arose from disagreement over how Anthropic's AI should be used, with the Department of Defense wanting unrestricted access while Anthropic wanted safeguards against fully autonomous weapons and domestic mass surveillance.

Anthropic, a company that makes Claude (an AI assistant), is suing the Department of Defense after the agency labeled it a "supply chain risk," which prevents other companies and government agencies from using Anthropic's AI models. The conflict started because Anthropic refused to give the Pentagon unrestricted access to its technology, citing concerns about mass surveillance of Americans and fully autonomous weapons that make targeting decisions without human input. Anthropic argues the DOD's actions violate free speech protections in the Constitution.

X has added a toggle in its iOS app that claims to block Grok (an AI chatbot) from editing your photos, but the feature has a major limitation. According to the fine print, it only prevents users from tagging @Grok in replies to your images on X, rather than actually stopping Grok from editing your photos.

Current US laws have not kept pace with AI capabilities, creating legal ambiguity around whether the government can conduct mass surveillance on Americans using AI systems. A dispute between the Department of Defense and AI company Anthropic has exposed this gap, with the White House responding by issuing new guidelines requiring AI companies to allow 'any lawful' use of their models, though questions about what is actually lawful remain unanswered.

Multi-LLM collaborative systems (setups where multiple AI models work together on complex tasks) can be attacked through three new methods: Decision Poisoning Attack (injecting false instructions to manipulate system output), Indirect Echoleak Attack (extracting private information through model interactions), and Information Collision Attack (exploiting communication between models). While these collaborative systems offer flexibility and better reasoning, their internal communication channels create security and privacy vulnerabilities that attackers can exploit.