AI Sec Watch

Members of Congress from both parties are pushing for guardrails (safety rules and oversight) on AI companies, with surveys showing most Americans support a new federal agency to monitor AI and require safety tests for critical decisions. President Trump dismissed these concerns as a hoax, but lawmakers like Democrat Don Beyer argue the government must regulate AI rather than letting companies regulate themselves, comparing the need to existing oversight in industries like medicine and automobiles.

A vulnerability was found in vLLM (a library for running large language models) version 0.27.1 and earlier, where attackers can manipulate the chat_template parameter to cause excessive resource consumption through Jinja template rendering (a system for dynamically generating text). The vulnerability can be exploited remotely, and a fix has been proposed but not yet officially accepted.

A financially motivated bug bounty hunter created PhantomRaven, a JavaScript-based information stealer (malware that collects sensitive data) distributed through npm, a popular platform where developers share code packages. The threat actor likely used an LLM to write the malware and deployed it via dependency-confusion attacks (tricking systems into downloading malicious packages instead of legitimate ones), though CrowdStrike's analysis suggests they use the stolen information only to identify bug bounty opportunities rather than selling it.

Samsung invested $231 million in Euclyd, a Dutch startup designing AI chips with a different architecture than Nvidia's GPUs (graphics processing units, specialized chips for processing data). Euclyd is developing chips specifically for inference (running already-trained AI models) and claims its systems will reduce energy use and costs for AI data centers, targeting commercial deployment starting in 2028.

Major AI company leaders including those from OpenAI, Anthropic, Google DeepMind, and SpaceX agreed to slow down AI development, citing safety reasons and proposing third-party auditors (independent evaluators who check whether systems are safe) and global regulations. Critics argue the agreement is actually an anticompetitive cartel (illegal cooperation between companies to limit competition) designed to block smaller competitors and the open-source community rather than improve safety.

Broadcom's CEO dismissed concerns that Anthropic's proposal to slow down frontier AI model development (the creation of increasingly powerful AI systems) would hurt the chipmaker's business, stating the company maintains its revenue forecasts for AI semiconductors through 2028. The slowdown proposal from Anthropic's CEO sparked a stock market sell-off among chip companies, but Broadcom's leader expressed confidence that demand for compute infrastructure (the hardware needed to run AI systems) and AI inference (using trained models to make predictions or generate outputs in real-world applications) will remain strong.

CrowdStrike CEO George Kurtz argues that slowing AI development won't reduce security risks because dangerous models are already widely available, including both frontier models (the most advanced AI systems) and open-weight models (publicly shared AI systems that anyone can download). He says the real solution is stronger AI-powered cybersecurity tools that can monitor and control AI agents (autonomous programs that make decisions independently) once they are deployed, rather than trying to prevent their development.

IBM Langflow OSS versions 1.0.0 through 1.10.0 have a critical vulnerability where attackers can run arbitrary Python code (code that does whatever the attacker wants) with root privileges (the highest access level) by uploading components that import socket or urllib libraries. This allows attackers to steal AWS credentials, steal files from the server, or attack other services like PostgreSQL and Redis running on the same network, while a faulty security check incorrectly marks these malicious components as safe.

President Trump called Nvidia CEO Jensen Huang at a tech conference to express support for AI data center development, calling concerns about data centers and AI a 'hoax' and praising them as 'the oil of the next 20, 25 years.' This followed Trump's criticism of Anthropic CEO Dario Amodei's argument that AI companies should intentionally slow down development to address safety concerns. The phone call highlights Trump's opposition to AI regulation and his alignment with major tech companies pushing for rapid AI advancement.