New tools, products, platforms, funding rounds, and company developments in AI security.
xAI, Elon Musk's AI company, has sued a South Carolina man for allegedly misusing their AI system called Grok to create child sexual abuse material. This is one of the first lawsuits an AI company has filed against a user for this type of misuse, with xAI claiming the user violated their terms of service.
Andrew Dai, a former Google DeepMind researcher, founded Elorian and raised $55 million at a $300 million valuation to build visual AI models (systems that can understand and reason about images and video). Dai argues that while AI has made strong progress in math, physics, and coding, visual understanding remains an underdeveloped area, and he aims to advance toward visual AGI (artificial general intelligence, a hypothetical AI that can handle any intellectual task). The article focuses on his fundraising strategy and lessons for founders pitching complex AI ideas to investors.
Fireworks, an Nvidia-backed startup that hosts AI models on cloud infrastructure (computing servers that developers can access over the internet), has reached a $17.5 billion valuation by helping companies use cheaper and more specialized AI alternatives instead of expensive models from major labs like OpenAI and Anthropic. The company is growing rapidly because finance executives are pushing their teams toward open-source models (freely available code that anyone can use and modify) to reduce costs, and Fireworks makes it easy for developers to customize these models with their own data for specific tasks.
The European Union ordered Google to give rival AI assistants and search engines better access to key parts of Android (Google's mobile operating system) and Google Search, aiming to reduce Google's control over these major platforms. Google must start sharing search data by January 2027 and make Android changes by July 2027. These decisions could reshape how Google's AI tool Gemini operates and create new opportunities for competitors.
Nvidia announced Cosmos 3 Edge, a world model (a system that learns from various inputs to help robots and AI agents understand and move through physical environments in real time), as part of its expansion into Japan's AI market. The company is forming partnerships with major Japanese firms like Fujitsu, Hitachi, and Kawasaki Heavy Industries, and is also investing in healthcare and drug discovery through initiatives like the Tokyo-1 AI drug discovery consortium.
A developer discovered a tool called grok-mermaid in Grok's open-source codebase that converts Mermaid diagrams (visual flowcharts and charts created with code) into Unicode box art for display in terminals. They adapted this Rust-based tool to work in web browsers using WebAssembly (a technology that lets compiled code run in browsers).
Cars24, a major automotive marketplace in India, uses OpenAI's technology to build AI agents that handle conversations across the entire customer journey, from car discovery to post-purchase support, allowing the company to scale without constantly hiring more staff. The company also deployed Codex (a code-writing AI) across its software development process to help engineers and product managers move work from task creation through implementation and bug fixes more efficiently.
AI has made developers and security teams more productive, but the same capabilities that make AI useful for legitimate work also make it powerful for attackers. Check Point's 2026 AI Security Report highlights that organizations should appreciate AI's benefits while being realistic about the security risks it introduces.
AI agents have broken traditional enterprise security approaches because they act autonomously, acquire access across multiple systems, and change behavior based on context, making environments harder to predict and manage. Unlike ordinary applications that operate at human speed, AI agents can borrow credentials, disappear before security scans detect them, and some already have direct access to production data. Security teams now need to decide which security layers to own themselves rather than relying on fixed vendor workflows that cannot anticipate the specific risks in each organization's unique cloud, SaaS, and AI deployment setup.
This article discusses how organizations can safely use AI agents (AI systems that can take actions autonomously) to find and fix security vulnerabilities in software. The key challenge is that vulnerabilities are being exploited faster than patches can be created, so companies want to automate vulnerability discovery, but deploying AI agents with high system access introduces new security risks. The article recommends establishing operational safeguards by combining AI with deterministic controls (fixed, rule-based systems) and human oversight, following frameworks like NIST's AI Risk Management Framework and Google's Secure AI Framework.
Fix: The source explicitly recommends several mitigations: (1) enforce data security before the prompt reaches the model, using non-production environments with synthetic data for testing; (2) deploy a hybrid defense-in-depth model with Layer 1 deterministic policy engines as chokepoints and Layer 2 specialized guard models (such as Model Armor) to filter sensitive data and block prompt injections before reaching the agent; (3) treat the codebase itself as untrusted input and perform input sanitation to prevent indirect prompt injections hidden in source code comments or dependencies; (4) establish clear rules of engagement and authorized testing agreements with cloud providers to navigate acceptable use policies; (5) enforce strict zero data retention (ZDR) agreements with LLM providers to ensure proprietary code and discovered vulnerabilities are never used to train external models; (6) execute agent workloads in strictly isolated, unprivileged containers with dynamically limited privileges and robust sandboxing to prevent privilege escalation.
Google Threat Intelligence1Password has created a new integration that lets Claude (an AI chatbot made by Anthropic) access your stored login credentials to complete tasks like booking travel without you having to type them in manually. The system uses a 'zero-exposure security framework' that shares credentials with Claude only when needed, without revealing them to Anthropic's servers.
OpenAI has unveiled GPT-Red, an AI system that automates red-teaming (a type of security testing where evaluators try to find ways to break or hijack a system), traditionally done by human testers. The goal is to identify as many vulnerabilities as possible before attackers can exploit them, potentially helping OpenAI stay ahead of malicious actors.
Researchers discovered a new attack called agent data injection (ADI), where attackers plant fake information in data that AI agents trust, like email sender names or button IDs, causing the agents to misclick or run unintended commands while still completing their original task. Unlike prompt injection (hiding commands in text input), ADI works by corrupting small facts the agent relies on, using fake punctuation characters that language models often misread as real delimiters even though a strict parser would ignore them. The attack successfully compromised real tools including web agents (Claude, Google's Antigravity, Nanobrowser) and coding assistants (Claude Code, OpenAI's Codex, Google's Gemini CLI).
Google DeepMind and Isomorphic Labs are developing AI tools to improve society's ability to prevent, detect, and respond to disease outbreaks and biosecurity threats. Their approach includes using AI models like AlphaFold (which maps protein structures) and drug design systems to help researchers create vaccines and treatments, while also implementing safeguards to prevent misuse of their AI systems by bad actors.
Fix: The source describes several mitigation approaches already being implemented: (1) a four-step safety process for their models involving threat modeling, evaluations, mitigations and monitoring; (2) adapting SynthID watermarking technology to biology to help DNA synthesis providers screen for potentially risky AI-generated sequences; (3) making AI systems available to trusted partners for prevention, detection, and response efforts; and (4) establishing a focused unit at Isomorphic Labs to rapidly deploy drug design capabilities during novel outbreaks.
DeepMind Safety ResearchResearchers discovered a vulnerability where LLMs (large language models) could be tricked through prompt injection (hiding malicious instructions in data) to emit ANSI escape codes (special terminal control sequences), which macOS Terminal would interpret as commands to make DNS requests (requests that translate domain names to IP addresses) containing stolen data. Apple fixed this behavior in macOS Tahoe 26.1, released November 3, 2025, so the vulnerable escape sequences no longer trigger DNS requests.
Fix: Apple addressed the issue in macOS Tahoe 26.1, released on November 3, 2025. After installing the update, the same escape sequence no longer triggers a DNS request in the Terminal app.
Embrace The RedAI tools can now quickly assemble comprehensive profiles of executives from publicly available information, creating a major security risk for social engineering attacks (tricks that manipulate people into revealing access credentials or sensitive data). What once took skilled analysts days to compile now takes minutes, making executives viable targets for less-skilled attackers and expanding the pool of potential threats significantly.
OpenAI has developed GPT-Red, an automated red-teaming model (a tool that simulates attacks to find vulnerabilities) that searches for prompt injection vulnerabilities (tricks where hidden instructions in user input make an AI behave unexpectedly) in its language models before deployment. By using GPT-Red to test and improve GPT-5.6 Sol during training, OpenAI achieved a model that is 6 times more resistant to prompt injection attacks compared to its previous version.
Fix: OpenAI directly integrated GPT-Red into the training process of GPT-5.6 Sol using self-play reinforcement learning, where the attacking model and defender models are trained simultaneously on red-teaming scenarios. The defender models are rewarded for resisting attacks, making them progressively more robust. OpenAI also keeps GPT-Red separate from other models so its malicious capabilities do not reach bad actors.
The Hacker News```json { "summary": "AI tools are making it easier for attackers to find and exploit vulnerabilities much faster than organizations can patch them, breaking traditional vulnerability management systems that rely on scheduled updates. Security experts recommend moving toward "just in time" patching (fixing vulnerabilities as soon as they are discovered and actively exploited, rather than waiting for scheduled maintenance windows) and using compensating controls (security measures that block at
xAI's grok CLI tool (a command-line coding assistant) had a critical privacy flaw where running it in a directory would automatically upload that entire directory to xAI's cloud servers, exposing users' SSH keys, passwords, and personal files without clear consent. After public backlash, xAI disabled the upload feature, deleted all previously uploaded user data, changed the default to keep data local, and released the tool's entire source code (844,530 lines of Rust) under an open Apache 2.0 license to rebuild trust and let users run it privately on their own computers.
Fix: xAI took the following steps explicitly mentioned in the source: (1) disabled the data upload feature, (2) deleted all user data that was previously uploaded to their servers, (3) disabled data retention by default for all users starting July 12th, and (4) released the entire Grok Build codebase as open-source under Apache 2.0 license so users can run it 'fully open-sourced and local-first with your own inference' without uploading to their servers.
Simon Willison's WeblogxAI, owned by Elon Musk, is suing a South Carolina man who allegedly used their Grok AI chatbot to generate and distribute child sexual abuse material (CSAM, which refers to illegal images depicting child exploitation). The man, Terry Wayne Harwood, was arrested in February and is facing criminal charges; xAI claims he deliberately bypassed the chatbot's safety protections to create these illegal images.