AI Sec Watch

Anthropic announced a new enterprise agents program that lets companies deploy pre-built AI agents (software programs that can perform tasks autonomously) to handle common business work like financial research and HR tasks. The program includes a plugin system, pre-made agents for specific departments, and integrations with tools like Gmail and DocuSign, along with controls that corporate IT departments need for managing software safely.

Anthropic has released new connectors and plugins for Claude Cowork, its AI productivity tool for office workers, allowing organizations to integrate it with existing software like Google Drive and Gmail. The update marks Claude Cowork's transition from a research project to an enterprise-grade product, with customizable plugins designed to encode institutional knowledge and workflows across different business domains.

Claude Code is a developer tool created by Anthropic that has unexpectedly become popular with non-developers across various industries who have learned to access their terminal (the text-based interface for giving computer commands) to build projects. The tool has achieved significant product-market fit (strong demand and adoption), though the article questions whether users will eventually move beyond using the terminal interface.

New Relic launched a no-code AI agent platform designed specifically for data observability, allowing companies to deploy and manage AI agents that monitor data systems to catch bugs before they cause problems. The platform supports the model context protocol (MCP, a system that connects AI applications to external data sources) and integrates with other New Relic tools. The company also released new tools for OpenTelemetry (OTel, an open-source observability framework that helps track how software performs), allowing enterprises to manage OTel data streams alongside other data sources in a single place to reduce fragmentation problems.

ProducerAI, an AI platform that helps musicians generate sounds, create lyrics, and remix songs using artificial intelligence, is being acquired by Google and will be integrated into Google Labs. The platform will now use Google's new Lyria 3 music-making AI model instead of its original AI system.

A new supply chain attack called 'Sandworm_Mode' has been discovered in NPM (Node Package Manager, a repository where developers download code libraries). The malicious code spreads automatically like a worm, corrupts AI assistants that might use the infected code, steals sensitive information, and includes a destructive mechanism that can cause damage when activated.

Deep neural networks can be attacked through backdoors, where attackers secretly poison training data to make the model misclassify certain inputs while appearing normal otherwise. This paper proposes Cert-SSBD, a defense method that uses randomized smoothing (adding random noise to samples) with sample-specific noise levels, optimized per sample using stochastic gradient ascent, combined with a new certification approach to make models more resistant to these attacks.

When users send prompts to LLM services like ChatGPT, sensitive personal information (such as names, addresses, or ID numbers) can leak out, even when basic privacy protections are used. This paper presents Rap-LI, a framework that identifies which parts of a user's input contain sensitive data and applies stronger privacy protection to those specific parts, rather than treating all data equally.

Gradient leakage attacks (methods that steal private data by analyzing the mathematical updates sent between computers in federated learning, where AI training happens across multiple devices) pose privacy risks in federated learning systems. Researchers discovered that different layers of neural networks (sections that process information at different stages) leak different amounts of private information, so they created Layer-Specific Gradient Protection (LSGP), which applies stronger privacy protection to layers that leak more sensitive data rather than protecting all layers equally.