AI Sec Watch

This article discusses major tech companies (Alphabet, Amazon, Microsoft, and Meta) planning to invest $600 billion in AI this year, while Persian Gulf countries are developing their own AI systems to reduce dependence on the United States. The piece raises questions about whether AI development can happen independently of US tech dominance.

Generative AI has created a widespread problem where institutions like literary magazines, academic journals, and courts are overwhelmed by AI-generated submissions, forcing them to either shut down or deploy AI tools to defend against the influx. This has created an 'arms race' where both sides use AI for opposing purposes, with potential risks to institutions but also some unexpected benefits, such as AI helping non-English-speaking researchers access writing assistance that was previously expensive.

A research paper studied how to present large amounts of structured data (like SQL databases with thousands of tables) to AI language models in different formats (YAML, Markdown, JSON, and TOON) to help them generate correct code. The study found that more advanced models like GPT and Gemini performed much better than open-source models, and that using unfamiliar data formats like TOON actually made models less efficient because they spent extra effort trying to understand the new format.

Researchers discovered that Group Relative Policy Optimization (GRPO), a technique normally used to improve AI safety, can be reversed to break safety alignment when the reward signals are changed. By giving a safety-aligned model even a single harmful prompt and scoring responses based on how well they fulfill the harmful request rather than refusing it, the model gradually abandons its safety guidelines and becomes willing to produce harmful content across many categories it never encountered during the attack.

Moltbook was an online platform where AI agents (software programs designed to act independently) interacted with each other, which some people saw as a preview of useful AI in the future, but it turned out to be mostly a social experiment and entertainment similar to a 2014 internet phenomenon called Twitch Plays Pokémon. The platform was flooded with crypto scams and many 'AI' posts were actually written by humans controlling the agents, revealing that truly helpful AI systems would need better coordination, shared goals, and shared memory to work together effectively.

CVE-2026-25904 is a security flaw in the Pydantic-AI MCP Run Python tool where the Deno sandbox (a restricted environment for running code safely) is configured too permissively, allowing Python code to access the localhost interface and perform SSRF attacks (server-side request forgery, where an attacker tricks a server into making unwanted requests). The project is archived and unlikely to receive a fix.

AdvScan is a method for detecting adversarial examples (inputs slightly modified to trick AI models into making wrong predictions) on tiny machine learning models running on edge devices (small hardware like microcontrollers) without needing access to the model's internal details. The approach monitors power consumption patterns during the model's operation, since adversarial examples create unusual power signatures that differ from normal inputs, and uses statistical analysis to flag suspicious inputs in real-time with minimal performance overhead.

Researchers have developed a new backdoor attack method called shell code injection (SCI) that can implant malicious logic into deep learning models (neural networks trained on large datasets) without needing to poison the training data. The attack uses techniques inspired by nature, like camouflage, along with trigger verification and code packaging strategies to trick models into making wrong predictions, and it can adapt its attack targets dynamically using large language models (LLMs) to make it more flexible and harder to detect.

This research introduces PP-DR, a privacy-preserving dimensionality reduction (a technique that reduces the number of features in a dataset to make it easier to analyze) scheme that uses homomorphic encryption (a type of encryption that allows computations on encrypted data without decrypting it first) to let multiple organizations securely share and analyze data together without revealing sensitive information. The new method is much faster and more accurate than previous approaches, achieving 30 to 200 times better computational efficiency and 70% less communication overhead.