AI Sec Watch

CrowdStrike is launching AI Discovery and Governance for Falcon for IT to help organizations find and control AI tools across their infrastructure, addressing the risk of shadow AI (unsanctioned AI systems and locally deployed models running without centralized oversight). Shadow AI expands the attack surface because these systems inherit existing permissions and can access data and credentials, but many organizations lack visibility into where AI is running or what it can access. The new capability gives security teams visibility into AI tools, local model runtimes, SDKs (software development kits, code libraries for building applications), and external AI service integrations at the endpoint layer, enabling them to discover, assess, and govern AI use.

A vulnerability called OS command injection (a flaw that lets attackers run unauthorized system commands) was found in the Bash Tool component of chatgpt-on-wechat software versions up to 2.0.8. The vulnerability exists in the _get_safety_warning function and can be exploited remotely, meaning an attacker doesn't need direct access to the affected system. This weakness has been publicly disclosed and could be actively exploited.

Fashion companies are increasingly using generative AI (machine learning technology that creates new images) to produce digital models and product imagery instead of hiring human models or photographers. One Australian retailer emphasized that AI-generated images should be clearly labeled and show products accurately, while a fashion designer noted that these tools can help small brands work more efficiently while maintaining quality standards.

Some wealthy tech leaders, including Sam Altman of OpenAI and Elon Musk, are promoting a 'transhuman' vision where humans and AI merge or AI becomes the dominant species. Altman warns that if humans and AI both compete for dominance, conflict could result, while Musk suggests humanity's main purpose is to create advanced digital intelligence.

Anthropic published documentation explaining how they use multiple containment techniques to restrict what Claude can do across their products. They use process sandboxes (isolated execution environments), virtual machines (complete simulated computers), filesystem boundaries (limiting file access), and egress controls (preventing unauthorized data transfer) to prevent AI agents from accessing credentials, exfiltrating data (stealing information), or reaching unintended systems, even if a user, the AI model, or an attacker tries to find workarounds.

Researchers have developed a technique called Model X-Ray that can detect hidden malware embedded in AI model weights (the numerical parameters that make up a trained AI system) using few-shot learning (training a detector with only a small number of examples). This work addresses a security risk where attackers could hide malicious code inside AI models that might go undetected during normal use.

Pope Leo XIV released a major teaching warning about AI's harms, including job displacement, accelerated warfare, and environmental exploitation. Anthropic co-founder Chris Olah spoke at the Vatican ceremony, which some experts criticize as potentially creating superficial 'feelgood' messaging rather than substantive critical examination of AI risks.

Researchers discovered Greyvibe, a Russia-aligned crime group that uses large language models (LLMs, AI systems trained to generate text) extensively throughout its cyberattacks against Ukrainian targets, including government and military organizations. The group has used generative AI to create spear phishing emails (fraudulent messages pretending to come from trusted sources), malicious scripts, and custom malware programs like PhantomRelay and LegionRelay (remote access trojans, or RATs, which are tools that let attackers control compromised computers). Greyvibe has conducted multiple campaigns since August 2025 using various attack methods, from fake websites to ClickFix-style attacks (tricks that convince users to run malicious commands on their computers).

PraisonAI (a framework for building AI agents) versions 4.6.37 and earlier have a vulnerability where hidden metadata in webpages can trick AI agents into writing files to any location on a system. The bug happens because the `write_file` function skips path validation (checking whether a file path is safe) when the workspace parameter is `None`, which is the default in production environments.