AI Sec Watch

Palantir continues using Anthropic's Claude (a large language model, or LLM, which is AI software trained to understand and generate text) despite the Pentagon designating Anthropic a supply-chain risk (a company or product deemed potentially unreliable or unsafe for government use). The Department of Defense plans to phase out Anthropic's tools over six months, though exemptions may be granted for critical national security operations.

Microsoft and other major tech companies filed legal briefs supporting Anthropic's court challenge against a Pentagon designation that blocks the AI company from government work. Microsoft argued that the restriction would disrupt suppliers who use Anthropic's AI tools, including those providing systems to the US military.

The ha-mcp OAuth consent form has a cross-site scripting (XSS) vulnerability, where user-controlled data is inserted into HTML without escaping (the process of converting special characters so they display as text rather than execute as code). An attacker could register a malicious application and trick the server operator into visiting a crafted authorization URL, allowing the attacker to run JavaScript in the operator's browser and steal sensitive tokens. This only affects users running the beta OAuth mode, not the standard setup.

Prompt abuse occurs when attackers craft inputs to make AI systems perform unintended actions, such as revealing sensitive information or bypassing safety rules. Three main types exist: direct prompt override (forcing an AI to ignore its instructions), extractive abuse (extracting private data the user shouldn't access), and indirect prompt injection (hidden malicious instructions in documents or web pages that the AI interprets as legitimate input). The article emphasizes that detecting prompt abuse is difficult because it uses natural language manipulation that leaves no obvious trace, and without proper logging, attempts to access sensitive information can go unnoticed.

Anthropic, maker of the AI assistant Claude, is in a legal dispute with the Pentagon after being designated a supply chain risk (a company that poses a security threat to government operations). The core issue involves disagreement over whether the U.S. government can be trusted to follow the law when using AI for surveillance, given a long history of government lawyers interpreting surveillance laws in ways that expand government monitoring far beyond what the plain language of those laws seems to allow.

Current popular AI video models like Sora, Vevo, and Runway aren't very effective for making films and TV shows, despite hype suggesting AI could create entire productions automatically. AI companies are now developing custom models designed specifically for filmmakers' creative needs while trying to avoid copyright issues.

The U.S. Department of Defense designated Anthropic's Claude AI as a supply chain risk, citing concerns that the company's built-in policy preferences (established through its constitutional training approach) could compromise military effectiveness and security. The Pentagon requires defense contractors to certify they don't use Claude, though the DOD acknowledged that transitioning away from the technology will take time.

This research paper explores vulnerabilities in Pedestrian Attribute Recognition (PAR), a computer vision task that identifies characteristics of people in images using AI models. The authors developed both adversarial attacks (methods to fool the system with manipulated images) and a defense strategy called semantic offset defense to protect PAR systems, testing their approach on multiple datasets.

This research paper presents a new method called FoVB (Forgery-aware Audio-Visual Adaptation with Variational Bayes) to detect deepfakes (AI-generated fake videos that manipulate both audio and video). The method works by analyzing the relationship between audio and video to find mismatches, such as when lip movements don't match the sound, which are telltale signs of deepfakes.