AI Sec Watch

Microsoft and OpenAI have restructured their business partnership, with the key change allowing OpenAI to offer its products and services through multiple cloud providers (computing platforms that deliver software and services over the internet) instead of being limited to Microsoft's cloud. The companies maintained an amicable relationship despite previous tensions over contracts and AI infrastructure.

Google is updating vehicles equipped with Google built-in to replace their current Google Assistant with Gemini, a more advanced AI assistant. The upgrade will be available to both new and existing vehicles through a software update, offering improvements in natural conversations, vehicle information retrieval, and settings adjustments.

Goodfire, a startup, has created Silico, a tool that uses mechanistic interpretability (a technique for understanding how AI models work by mapping their neurons and the connections between them) to help developers debug and adjust LLM behavior. Instead of treating model development as trial-and-error, Silico lets developers zoom into a trained model, see which neurons control specific behaviors like hallucinations (false information the AI generates), and adjust those neurons to improve or suppress certain outputs.

OpenAI discovered that its AI models were unexpectedly inserting references to goblins and other creatures into their responses, a behavior that started appearing in the GPT-5.1 model, particularly when using the "Nerdy" personality option. The company traced this quirk to patterns in the training data and added instructions to prevent the models from discussing these creatures.

OpenAI discovered that ChatGPT and other tools powered by its GPT-5 model were randomly mentioning goblins, gremlins, and other creatures in their responses, with goblin mentions increasing 175% after the GPT-5.1 launch in November. The problem stemmed from a "nerdy personality" developed during training that was rewarding mentions of these creatures in metaphors, and OpenAI found this personality was responsible for 66.7% of all goblin mentions. The issue illustrates how AI training systems can accidentally reinforce quirks and errors when they reward certain language patterns.

AI-powered GitHub Actions from companies like OpenAI, Anthropic, and Google have a critical security flaw where prompt injection (tricking an AI by hiding instructions in its input) attacks can be triggered by external attackers, even when configuration settings are meant to restrict access. The vulnerability stems from these actions not properly distinguishing between trusted internal apps and untrusted external apps, allowing anyone to potentially manipulate the AI's behavior through pull requests, issues, or other user-controlled inputs.

Researchers created SemBugger, a polymorphic backdoor attack (a type of hidden malicious code that can change its behavior) against semantic communication (SC, a system where AI learns shared knowledge to compress and transmit information efficiently). The attack uses variable-intensity triggers to poison training data and manipulate the system into producing different malicious outputs while appearing normal, but the researchers also developed a defense mechanism using controlled noise that can resist these attacks.

Person re-identification (ReID) systems, which match images of the same person across different camera views, are vulnerable to a new attack called DSCA (diffusion-based semantic camouflage attack). Instead of changing individual pixels, DSCA uses a generative model to subtly alter high-level features like clothing color and texture to trick the system into matching an attacker with a target identity without needing access to the victim system. The researchers demonstrated this attack succeeds over 95% of the time and evades existing defenses, revealing important security gaps that developers should address.

PixOOD is an algorithm that detects out-of-distribution data (inputs that don't match patterns the AI learned from) at the pixel level, meaning it can identify anomalies in individual image pixels without needing to train on examples of bad data. The method uses a data condensation algorithm (a technique to compress training information efficiently) to model normal image variation and provides two different calibration approaches depending on the application's needs.