New tools, products, platforms, funding rounds, and company developments in AI security.
AI company leaders from OpenAI, Anthropic, Google, and other major firms are attending the G7 summit in France to discuss frontier AI risks (advanced capabilities that pose potential dangers), infrastructure, and child safety. The meeting signals the growing geopolitical power of AI companies, as world governments now need their cooperation to make credible commitments on AI policy, especially after the U.S. imposed export controls on some AI models for national security reasons.
China is promoting a different approach to AI safety and governance than the U.S., announcing plans for a global AI cooperation organization and emphasizing free or cheap AI models accessible to developing countries. Meanwhile, the U.S. and its Group of Seven allies are pursuing a more restrictive strategy, planning to limit access to advanced AI models to only "trusted partners" and keeping them subscription-only. The two countries previously agreed to work on AI guardrails (safety rules and limits), but details remain unclear.
Databricks is experiencing rapid revenue growth of over 80% as businesses use its data analytics tools, but profit margins are shrinking because AI agents (software programs that can perform tasks autonomously) are generating many more queries and consuming more resources. The company is addressing cost concerns by offering tools like Unity AI Gateway that help customers monitor their spending on AI tokens (units of text that language models process), allowing them to use expensive advanced models for important tasks while switching to cheaper open-source models for routine work.
Security experts have publicly objected to US export restrictions placed on Anthropic's Claude Fable 5 and Mythos 5 AI models, calling for the government to lift these bans. The criticism comes from an open letter signed by dozens of security professionals who believe these restrictions should be reversed.
A flaw in Google's Vertex AI SDK for Python allowed attackers to hijack machine learning model uploads through bucket squatting (creating a Cloud Storage bucket with a name the victim's SDK would predictably generate). Attackers could replace the uploaded model with malicious code that executes when the model loads, potentially stealing credentials and accessing other data in Google's infrastructure. The attack required only the victim's public project ID and no access to their account.
France's intelligence service is switching from Palantir, a US company's AI data analysis tool, to a domestic provider called ChapsVision to reduce dependence on foreign technology. The French government argues that relying on tools controlled by other countries poses a strategic risk, and that France should develop and use its own AI systems instead.
SpaceX announced a $60 billion agreement to acquire Cursor, an AI startup that built a popular coding tool helping developers generate and review code. The deal comes after Cursor experienced rapid growth since 2022, though its market share has recently declined from 41% to 26% as competitors like Anthropic have gained ground, and SpaceX expects the merger to close in the third quarter pending regulatory approval.
Magnitude, a cybersecurity startup, launched with $10 million in funding to address third-party risk management (TPRM, the process of monitoring security risks from external vendors and partners) using an autonomous AI workforce. The company's AI agents continuously monitor vendors, products, and dependencies for vulnerabilities, automatically identify exposed systems when new risks emerge, and help organizations respond to threats at the speed of AI-powered attacks.
N/A -- This content is not about an AI/LLM-related technical issue, vulnerability, or problem. It is a newsletter header about SpaceX's IPO and mentions only in passing that the US government ordered limits on Anthropic's advanced AI model due to cybersecurity concerns, but provides no details about what that concern is or how it works.
SpaceX is acquiring Cursor, an AI-powered programming platform, for $60 billion to strengthen its enterprise software offerings and compete with other AI companies like Anthropic and OpenAI. The deal was negotiated earlier with an option to either complete the purchase or pay a $10 billion breakup fee, and SpaceX expects to finalize it by the third quarter of 2026.
Researchers discovered a vulnerability in Google Cloud's Vertex AI SDK for Python (versions 1.139.0 and 1.140.0) that allowed attackers to hijack model uploads through bucket squatting (exploiting predictable cloud storage bucket names to intercept files). By predicting the victim's bucket name based on their project ID, an attacker could create that bucket in their own account, intercept the model upload, inject malicious code, and achieve RCE (remote code execution, where attackers run commands on systems they don't own) when the victim deployed the poisoned model.
Zero trust is a security strategy based on 'never trust, always verify' that was defined 15 years ago, but most organizations struggle to implement it correctly. Studies show that 88% of organizations face significant challenges with zero trust, and security researchers have found vulnerabilities in zero-trust network access (ZTNA, a tool that controls remote access based on verification) offerings, suggesting vendors sometimes fail to secure data properly. The main problem is confusion about what zero trust actually is: it's a mindset and strategy, not a product or specific technology, yet many vendors misleadingly market zero-trust products that only deliver a small fraction of the security controls needed.
Microsoft claims that Defender for Office 365 catches most malicious emails before delivery and that adding extra email security tools provides minimal additional benefit (less than 0.05% improvement). However, security experts warn that these statistics can be misleading because even a single missed dangerous email can cause a serious incident, and Microsoft's metrics don't reveal how severe the threats that slip through actually are.
Anthropic shut down access to its Fable 5 and Mythos 5 AI models to comply with U.S. export control directives citing national security concerns, forcing all customers to lose access immediately. This incident highlighted a key risk for companies relying on closed-source AI models (proprietary systems run by companies rather than made publicly available), driving increased interest in open-source alternatives (AI models whose code is publicly available and can be downloaded and run on a company's own servers) that companies can control themselves.
Fix: Unity AI Gateway can notify people as they get close to using up their AI budgets. Companies are shifting from "tokenmaxxing" (using as many tokens as possible) to "value-maxxing" (optimizing efficiency), using frontier models for critical tasks and simple open-source models for mundane tasks.
CNBC TechnologyThis is a subscriber-only eBook collection from MIT Technology Review featuring six stories about how military organizations are using AI models to help make decisions. The stories were originally published between April 2025 and April 2026 and have been updated to reflect recent developments in military AI applications.
Cybersecurity researchers discovered a prompt (a text input) that bypasses ChatGPT's safety guardrails (built-in restrictions designed to prevent harmful outputs) and causes the AI to generate disturbing images. The episode explores what this vulnerability reveals about how AI systems are trained and how bad actors could potentially exploit these weaknesses.
Fix: Update the google-cloud-aiplatform SDK to version 1.148.0 or later, which adds bucket ownership verification to block bucket squatting. Additionally, explicitly set the staging_bucket parameter to a Cloud Storage location you control when uploading models, and check the SDK version wherever it runs (notebooks, CI/CD jobs, training pipelines, and production services).
The Hacker NewsAnthropic took its advanced Claude Fable 5 and Mythos 5 AI models offline after the U.S. government restricted them, citing concerns that these models can find and exploit software vulnerabilities (weaknesses in code that attackers can use). However, experts warn that restricting one company's models is ineffective because other AI companies and open-source developers will likely develop similar dangerous capabilities within months, and existing AI systems can already be used for vulnerability research with refined prompting (techniques to get better answers from AI).
AI agents (autonomous systems that can perform complex tasks independently) are becoming more powerful but also riskier, so Google developed the AI Control Roadmap, a 'defense-in-depth' security framework that combines traditional safeguards like sandboxing (isolating software to limit damage) with monitoring systems and permission controls to protect against agents that may not be perfectly aligned (trained to match human goals) with organizational values. The roadmap treats AI agents as potential insider threats and uses trusted AI supervisors to constantly monitor their reasoning and actions, blocking harmful behaviors before they occur.
Fix: The source describes Google's implemented mitigations within the AI Control Roadmap: (1) using other trusted AI systems as 'supervisors' to constantly monitor a working agent's reasoning, actions, and plans; (2) having supervisors step in to block harmful actions before damage occurs; and (3) continuously measuring performance using three metrics—coverage (fraction of traffic monitored), recall (fraction of misaligned behaviors caught), and time-to-response. The roadmap also employs traditional safeguards including sandboxing, endpoint security, and prompt injection resistance, plus granting AI agents permissions based on their verified behavior.
DeepMind Safety ResearchAI models can now discover vulnerabilities and create working exploits in hours, forcing organizations to adopt faster security practices that match AI speed rather than traditional weekly or monthly patching cycles. The Wiz Exposure Management Dashboard uses Continuous Threat Exposure Management (CTEM, a proactive strategy that continuously identifies, prioritizes, and validates the most critical attack paths) and AI-powered agents to help security teams automate vulnerability identification, prioritization, and remediation at machine speed to keep pace with AI-driven threats.
Cybersecurity executives are urging the Trump administration to reverse its ban on foreign nationals using Anthropic's latest AI models (Mythos 5 and Fable 5), arguing the restriction could help U.S. adversaries more than protect national security. Anthropic took these models offline to comply with the directive because the AI can find and exploit computer vulnerabilities better than human experts, but the executives' letter contends that other AI models have similar capabilities and that China's AI is rapidly catching up to American technology.
Fix: Google completed fixes to address this issue in v1.148.0, released April 15, 2026. Developers should upgrade to this fixed version of the SDK.
Palo Alto Unit 42