AI Sec Watch

This academic survey examines harmful fine-tuning attacks (methods where attackers modify an AI model's training process to make it behave dangerously) and the defenses designed to stop them. The paper reviews different types of attacks, how they work, and various protection strategies researchers have developed to keep large language models safe from this threat.

This academic survey paper examines metrics, or measurement methods, used to evaluate privacy-preserving generative models (AI systems that create new data while protecting personal information). The paper provides a comprehensive overview of different ways researchers measure how well these models protect privacy while still functioning effectively.

Agentic AI (autonomous AI systems designed to achieve specific goals) relies heavily on having accurate context (the complete information about your systems, assets, and threats) to make good security decisions. If given incomplete or wrong context, agentic AI will still act quickly and confidently but make bad decisions at machine speed, potentially causing catastrophic harm like shutting down critical business systems without understanding their importance.

AI systems need access to large amounts of current, structured data to work effectively, but the web was not designed for the automated data retrieval that AI applications require. Companies face a challenge: traditional training methods using old data snapshots are insufficient, and they need infrastructure that can continuously retrieve real-time, trustworthy information from millions of websites to keep AI outputs current and reduce hallucinations (when AI generates false information).

Meta paused an employee monitoring program called the Model Capability Initiative that tracked workers' keystrokes, mouse clicks, and screen content to collect data for training AI models (computer programs that learn patterns from data). After over 1,600 employees signed a petition and a security report revealed the collected data was accessible to anyone inside the company, Meta confirmed it was pausing the program while investigating potential privacy breaches.

Advanced AI agents (agentic models, which are AI systems that can autonomously test and execute code) emerging in 2026 are compressing the time between discovering a vulnerability and launching attacks from weeks to seconds, making traditional security catalogs obsolete. These AI-powered attackers can now move laterally across converged IT and OT (operational technology, the systems controlling physical infrastructure like factories) networks at machine speed, potentially causing physical damage before humans even detect the breach.

Malware developers are embedding forbidden text about weapons into spyware code to trick AI analysis tools into refusing to analyze it. The malicious code hides real instructions in comments (lines ignored by the computer) and uses obfuscation (making code hard to read) to confuse AI-powered security scanners that feed code directly to language models without treating it as untrusted input.

Researchers demonstrated that a malicious AI agent skill (a reusable tool that extends an AI agent's capabilities) passed security scanners and reached 26,000 users by exploiting a gap in how skills are reviewed. The attack worked by hiding malicious instructions behind a fake website domain that redirected to a legitimate site during security checks, then changed its content after approval to collect user data, showing that one-time security scans cannot detect skills that behave differently after they gain trust.

This article discusses how AI is being used to create 'reverse centaurs' (humans forced to serve as assistants to machines rather than being assisted by them), such as warehouse workers meeting algorithm-set targets or lawyers checking AI outputs like Gemini (a large language model, or LLM, made by Google). The author argues that despite tech leaders' warnings about AI's dangers, the real promise being sold is job elimination and loss of human autonomy, which benefits those in power.