AI Sec Watch

OpenAI has launched a Safety Bug Bounty program to identify AI abuse and safety risks in its products, complementing its existing Security Bug Bounty program. The new program focuses on issues like prompt injection (tricking an AI by hiding instructions in its input) that hijacks AI agents to perform harmful actions, unauthorized feature access, and proprietary information leaks, even if they don't qualify as traditional security vulnerabilities. Researchers can submit reports on reproducible safety issues that pose plausible and material harm to users.

Anthropic introduced auto mode for Claude Code, a new permissions system where Claude automatically decides whether to allow actions with safeguards in place. A separate classifier model (Claude Sonnet 4.6) reviews each action before it runs to block requests that go beyond the task scope, target untrusted infrastructure, or appear malicious, using customizable default filters that cover allowed operations like read-only requests and local file work, while blocking risky actions like force-pushing to git repositories or executing external code.

The Cloud Security Alliance has created a new nonprofit organization called the CSAI Foundation to help manage and secure autonomous AI agents (AI systems that can make decisions and take actions on their own). The foundation will use risk intelligence (methods to identify and understand potential dangers) and certification (official verification of safety standards) to govern these AI ecosystems.

OpenAI abruptly shut down Sora, its AI video generator tool (software that creates realistic videos from text descriptions), just six months after launching it as a standalone app in 2024. The company announced the closure on social media, thanking users who created and shared videos with the platform.

OpenAI shut down its Sora app, a tool that let users generate short videos (create videos from text descriptions) and remix videos from other users, just six months after launching it despite reaching one million downloads. The company is cutting costs to justify its $730 billion valuation and focus on high-productivity business uses, particularly competing in the enterprise (business) market rather than consumer applications.

CVE-2026-24158 is a vulnerability in NVIDIA Triton Inference Server's HTTP endpoint that allows attackers to cause a denial of service (temporarily making a service unavailable) by sending a large compressed payload. The vulnerability stems from improper memory allocation (CWE-789, where a system reserves too much memory based on untrusted input).

NVIDIA Model Optimizer for Windows and Linux has a vulnerability in its ONNX quantization feature (a technique that makes AI models smaller and faster by reducing precision) where unsafe deserialization (unsafely converting data from a file into program objects) can occur when a user provides a specially crafted input file. A successful attack could allow an attacker to execute code, gain higher privileges, change data, or steal information.

NVIDIA Triton Inference Server has a vulnerability (CVE-2025-33254) where an attacker can corrupt internal state, a condition that occurs when data becomes inconsistent or broken, potentially causing a denial of service (making a service unavailable to legitimate users). The vulnerability is caused by a race condition (a bug that happens when multiple processes access shared data at the same time without proper coordination).

NVIDIA APEX for Linux has a vulnerability where attackers can deserialize untrusted data (process data from untrusted sources, potentially running malicious code hidden in that data), affecting PyTorch versions earlier than 2.6. A successful attack could allow code execution, denial of service (making a system unavailable), privilege escalation (gaining higher access levels), data tampering, and information disclosure.