AI Sec Watch

Prompt abuse occurs when attackers craft inputs to make AI systems perform unintended actions, such as revealing sensitive information or bypassing safety rules. Three main types exist: direct prompt override (forcing an AI to ignore its instructions), extractive abuse (extracting private data the user shouldn't access), and indirect prompt injection (hidden malicious instructions in documents or web pages that the AI interprets as legitimate input). The article emphasizes that detecting prompt abuse is difficult because it uses natural language manipulation that leaves no obvious trace, and without proper logging, attempts to access sensitive information can go unnoticed.

Anthropic, maker of the AI assistant Claude, is in a legal dispute with the Pentagon after being designated a supply chain risk (a company that poses a security threat to government operations). The core issue involves disagreement over whether the U.S. government can be trusted to follow the law when using AI for surveillance, given a long history of government lawyers interpreting surveillance laws in ways that expand government monitoring far beyond what the plain language of those laws seems to allow.

Current popular AI video models like Sora, Vevo, and Runway aren't very effective for making films and TV shows, despite hype suggesting AI could create entire productions automatically. AI companies are now developing custom models designed specifically for filmmakers' creative needs while trying to avoid copyright issues.

The U.S. Department of Defense designated Anthropic's Claude AI as a supply chain risk, citing concerns that the company's built-in policy preferences (established through its constitutional training approach) could compromise military effectiveness and security. The Pentagon requires defense contractors to certify they don't use Claude, though the DOD acknowledged that transitioning away from the technology will take time.

This research paper explores vulnerabilities in Pedestrian Attribute Recognition (PAR), a computer vision task that identifies characteristics of people in images using AI models. The authors developed both adversarial attacks (methods to fool the system with manipulated images) and a defense strategy called semantic offset defense to protect PAR systems, testing their approach on multiple datasets.

This research paper presents a new method called FoVB (Forgery-aware Audio-Visual Adaptation with Variational Bayes) to detect deepfakes (AI-generated fake videos that manipulate both audio and video). The method works by analyzing the relationship between audio and video to find mismatches, such as when lip movements don't match the sound, which are telltale signs of deepfakes.

Microsoft launched Copilot Health, a feature that lets users ask an AI assistant questions about their medical records, lab results, and data from wearables (devices that track health metrics like heart rate) in a dedicated secure space within Copilot. The feature is rolling out gradually through a waitlist and is designed to help users understand their health data rather than replace doctors or provide medical diagnoses.

Google developed a flash flood prediction system by using Gemini (an LLM, or large language model) to analyze 5 million news articles and extract data about 2.6 million floods, creating a dataset called Groundsource. This dataset trained a machine learning model (LSTM, a type of neural network) that now provides flood risk forecasts for urban areas in 150 countries on Google's Flood Hub platform, though it has limitations like lower resolution than traditional weather services.

Google is adding an AI-powered feature called "Ask Maps" to Google Maps that uses Gemini (Google's AI assistant) to answer complex, specific questions about locations. Previously, Google Maps couldn't handle very detailed queries like "where can I charge my phone without waiting in line," but now Gemini can provide personalized, detailed answers to these kinds of questions.