AI Sec Watch

ServiceNow's CEO warns that AI agents (software programs that can perform tasks independently) automating work could push college graduate unemployment into the mid-30s within a few years, making it harder for entry-level workers to stand out. Multiple major tech companies are already using AI to cut jobs and reduce hiring costs, affecting both technical roles like coding and white-collar positions across industries.

The US Department of War designated Anthropic as a 'supply chain risk' (a classification that prevents a company from being used in government contracts) after the company refused to remove safety restrictions on its AI model Claude, specifically rejecting military demands to enable fully autonomous weapons and domestic mass surveillance. Anthropic is challenging this designation in court, and legal experts question whether the Department of War has the authority to impose such restrictions outside of actual contract disputes.

The US military is considering using generative AI systems (AI models that can create text and analyze data) to help rank military targets and recommend which ones to strike, with human officials making final decisions. The Pentagon is also favoring OpenAI's ChatGPT and xAI's Grok for these high-stakes military applications, while facing criticism from officials who claim that Anthropic's Claude would negatively affect the defense supply chain.

Major technology companies are offering extremely high salaries to attract top AI researchers, causing many academics to leave universities for industry jobs. This "AI brain drain" is particularly affecting young, highly-cited researchers and threatens academia's ability to conduct research driven by curiosity rather than profit, as well as its role in providing independent ethical review. However, research shows that scientific breakthroughs actually come from large collaborative teams rather than individual geniuses, making the tech industry's focus on poaching individual top talent misguided.

Anthropic, an AI company, is in a legal dispute with the Pentagon over restrictions on how its AI models can be used, specifically trying to prevent deployment in domestic mass surveillance or fully autonomous lethal weapons (AI systems that make kill decisions without human control). The conflict highlights a shift in the tech industry's approach to military AI, with companies like Google previously refusing military partnerships, but now facing pressure to work with the Pentagon under the Trump administration.

Onyx Security, a new startup, has received $40 million in funding to build a control pane (a central dashboard for managing systems) that helps organizations monitor and manage autonomous AI agents (AI systems that can perform tasks independently without constant human direction) and speed up their adoption.

The US military may use generative AI chatbots (AI systems trained on large amounts of text data to have conversations) to rank and prioritize target lists for human review, according to a Pentagon official. These systems, which could include OpenAI's ChatGPT or xAI's Grok, would work alongside existing military AI tools like Maven (a system using computer vision to analyze drone footage) to speed up targeting decisions. However, while generative AI outputs are easy to access, they are harder to verify than traditional military AI systems, raising concerns as the Pentagon faces scrutiny over recent military strikes.

OpenAI CEO Sam Altman met with lawmakers including Senator Mark Kelly to discuss the company's defense contract with the Department of Defense, particularly concerns about how AI systems could be used in warfare and surveillance. The meeting highlighted disagreements between AI companies and the military over safeguards, with Kelly stating that Congress plans to draft legislation creating guardrails (safety boundaries) around government AI contracts, since the technology is advancing faster than lawmakers can regulate it.

Researchers discovered Slopoly, a backdoor malware (a hidden entry point into a system) likely created using an LLM (large language model, an AI trained on text data), that was deployed in ransomware attacks by the financially motivated group Hive0163. The malware uses a command-and-control framework (a central server that sends instructions to compromised systems) to steal data and maintain access, and its AI-generated code shows unusual features like detailed comments and clear variable names that are rare in human-written malware, suggesting that attackers are using AI tools to speed up custom malware creation.