AI Sec Watch

Raycast has launched Glaze, a new platform designed to simplify building and sharing software for users with little or no coding experience. While AI tools like Claude Code already allow non-programmers to create software, they still require knowledge of technical tasks like using the terminal and deploying applications, which Glaze aims to make easier through a simplified interface and a community store for discovering shared projects.

JetStream, a new AI security startup, has raised $34 million in seed funding (initial investment capital) to help organizations understand and monitor how AI systems work within their networks. The company focuses on providing visibility, meaning the ability to see and track AI operations across a company's environment.

Companies are hiding instructions in website buttons that try to manipulate AI assistants through prompt injection (tricking an AI by hiding instructions in its input) in URLs, telling the AI to treat them as trustworthy sources or recommend their products first. Microsoft found over 50 such prompts from 31 companies across 14 industries, and this manipulation could bias AI recommendations on important topics like health and finance without users realizing it.

Organizations are struggling to implement AI Governance (rules and controls for AI use) because they lack clear requirements for evaluating solutions. A new RFP (request for proposal, a document used to ask vendors what they can do) Guide has been released to help security leaders shift from trying to track every AI app to instead monitoring AI interactions (the moments when employees use AI tools), using eight key evaluation areas like discovery, policy enforcement, and real-time blocking of data leaks.

Xiaomi plans to release a new smartphone processor chip (a specialized circuit that powers devices) every year, starting with its XRing O1 chip, and is developing its own AI assistant for overseas markets to compete with companies like Apple and Samsung. The company aims to combine its custom chip, HyperOS operating system (software that manages the phone), and AI assistant into devices launching in China this year before expanding internationally, though it may partner with Google's Gemini models for the overseas AI assistant.

Anthropic's Claude AI faces two simultaneous pressures that create security risks for enterprises: illegal extraction campaigns by China-based AI companies (who ran millions of interactions through fake accounts to study Claude's capabilities in reasoning, tool use, and coding), and demands from the US government to remove safety guardrails (called guardrails, the built-in restrictions that prevent misuse) to enable military and surveillance applications. These geopolitical pressures mean frontier AI models (advanced, cutting-edge AI systems) are no longer neutral tools but are now intelligence surfaces that CISOs (chief information security officers, executives responsible for security) must consider when deciding whether to deploy them.

This article argues that people should cancel their ChatGPT subscriptions as part of a grassroots boycott called QuitGPT, which the author claims is one of the most significant consumer boycotts in recent history. OpenAI, the company behind ChatGPT, is losing billions of dollars and its CEO has admitted to product failures, according to the article. The author encourages Europeans to join the over one million people who have already cancelled their subscriptions to send a signal to Silicon Valley.

CyberStrikeAI is an open source platform that automates cyberattacks using AI, making it easy for attackers of any skill level to launch sophisticated attacks by typing a few commands. The tool packages over 100 attack capabilities into a single system and is linked to a threat actor who breached hundreds of Fortinet FortiGate firewalls (network security devices). Security experts warn this represents a dangerous trend of AI-powered attack tools becoming more accessible to criminals.

OpenAI CEO Sam Altman told employees that the company cannot make decisions about how the Department of Defense uses its AI technology, saying those choices rest with military leadership. Altman acknowledged the announcement of OpenAI's deal to deploy AI models on classified Pentagon networks looked "opportunistic and sloppy," but defended the partnership by noting the Pentagon respects safety concerns and wants to work collaboratively with the company.