AI Sec Watch

Large language models (LLMs, AI systems trained on text data) are very bad at generating passwords because they create predictable patterns instead of truly random ones. The study found that Claude, an LLM, always started passwords with an uppercase G followed by 7, avoided repeating characters, never used the * symbol, and repeated the same password 36% of the time across 50 attempts. This is a serious problem because autonomous AI agents (AI systems that act without human control) will need to create accounts and authenticate themselves, but the passwords they generate are weak and easy to crack.

RSA 2026 will focus on five cybersecurity trends, including AI-SOCs (security operations centers using autonomous agents to handle alert triage and incident response), CTEM (continuous threat exposure management, which gives organizations a complete view of their assets and vulnerabilities to prioritize risk), and cyber resilience (the ability to anticipate, withstand, recover from, and adapt to attacks). Security leaders should approach these trends with cautious skepticism, asking tough questions about vendor claims and ensuring strong data foundations before adopting new tools.

Google API keys that were originally created as public identifiers for Google Maps became dangerous security risks when Google enabled the Gemini API on the same projects, because Gemini keys can access private files and make billable requests, yet developers were never notified of this privilege change. Truffle Security discovered nearly 3,000 exposed API keys in web archives that could access Gemini, including some belonging to Google itself, highlighting how a service upgrade unexpectedly transformed harmless public keys into secret credentials.

Nvidia CEO Jensen Huang argued that markets are wrong to fear AI agents will destroy software companies, saying instead that AI agents are 'tool users' that will rely on existing enterprise software tools like Excel, ServiceNow, and SAP to become more productive. Huang's comments came after Nvidia reported strong earnings and raised its revenue forecast, though some analysts warn that certain software companies could still face serious challenges as AI automates workflows and lowers barriers for new competitors.

Nvidia CEO Jensen Huang downplayed concerns about a dispute between the U.S. Defense Department and Anthropic, a company that makes Claude (a large language model, or LLM). The disagreement centers on whether Anthropic's AI tools can be used for autonomous weapons (weapons that make decisions without human control) and mass surveillance, with the Defense Department demanding unrestricted use while Anthropic seeks limitations.

Langflow, a tool for building AI-powered agents and workflows, had a vulnerability in versions before 1.8.0 where the CSV Agent node automatically enabled a dangerous Python execution feature. This allowed attackers to run arbitrary Python and operating system commands on the server through prompt injection (tricking the AI by hiding instructions in its input), resulting in RCE (remote code execution, where an attacker can run commands on a system they don't own).

Gushwork, an India-founded startup, is helping businesses get discovered through AI-powered search tools (systems like ChatGPT and Perplexity that use artificial intelligence to answer questions) by automatically creating search-optimized content and building backlinks (links from other websites that point to a business's site). The company raised $9 million in funding and reports that AI-driven search and chat platforms now account for about 40% of inbound leads for its customers, despite representing only 20% of website traffic.

A Chinese internet activist accidentally exposed details about coordinated political influence operations (organized campaigns to manipulate public opinion) that used ChatGPT to create negative content about Japan's Prime Minister Takaichi. The leak revealed how ChatGPT was being used as a tool to generate misleading material for political purposes.

Anthropic acquired Vercept, an AI startup that built tools for agentic tasks (AI systems that can independently perform complex actions), including a product called Vy that could control remote computers. Vercept's product will shut down on March 25, with some co-founders joining Anthropic while others, including investor Oren Etzioni, expressed disappointment about the acquisition ending the startup after just over a year.