AI Sec Watch

Apache OpenNLP has a vulnerability where three methods in AbstractModelReader read count values from binary model files without checking if they're reasonable, allowing an attacker to trigger an OOM error (a crash caused by the program running out of memory) by creating a malicious .bin file with an extremely large count value. This denial of service (making a service unavailable) attack requires minimal file size and crashes the Java virtual machine early during model loading.

Evolver, a self-evolving engine for AI agents, had a prototype pollution vulnerability (a bug where attackers inject malicious properties into core JavaScript objects) in versions before 1.69.3. The flaw existed in functions that merged user data without blocking dangerous keys like __proto__ and constructor, allowing attackers to modify how all JavaScript objects behave.

Evolver, a tool that helps AI agents improve themselves, had a command injection vulnerability (a security flaw where attackers trick the system into running unauthorized commands) in versions before 1.69.3. The flaw was in the _extractLLM() function, which built shell commands using simple string concatenation without cleaning the input first, allowing attackers to execute arbitrary commands on the server when certain input contained shell metacharacters (special characters that have meaning to the command system).

Evolver, a GEP-powered self-evolving engine for AI agents, contained a path traversal vulnerability (a type of attack where an attacker manipulates file paths to access files outside their intended directory) in versions before 1.69.3. The vulnerability was in the skill download command's --out= flag, which did not validate user-provided file paths, allowing attackers to write files to any location on the system, potentially overwriting critical files.

Anthropic has partnered with Goldman Sachs, Blackstone, and other investment firms to create a $1.5 billion venture that will deploy Claude, Anthropic's AI model, directly into businesses. The partnership aims to address a shortage of experts who can implement AI technology in real-world business operations by embedding engineers inside companies to redesign workflows and integrate AI into core processes, starting with companies owned by the investment firms.

A study found that AI platforms disproportionately reference Nigel Farage and Reform UK more than other UK political leaders when answering questions about British politics. Researchers suggest this indicates Reform UK has achieved unusual visibility in LLMs (large language models, AI systems trained on text data to generate responses).

Elon Musk is suing OpenAI and CEO Sam Altman in federal court, claiming he invested millions expecting OpenAI to remain a nonprofit organization but alleges the company was secretly converted into a for-profit corporation, deceiving him about its original mission. The trial centers on whether Musk was actually deceived and when he discovered this alleged misconduct, with Musk seeking damages and the reversal of OpenAI's restructuring that reduced the nonprofit portion's control.

Elon Musk, who co-founded OpenAI in 2015, is suing the company for allegedly breaking its commitment to remain a nonprofit and pursue a charitable mission, claiming they instead commercialized the AI technology. Two days before the trial started, Musk texted OpenAI's president Greg Brockman about settling the case, but when Brockman suggested both sides drop their claims, Musk responded with a threat about making him and CEO Sam Altman "the most hated men in America."

Researchers discovered a new type of denial-of-service attack called ThinkTrap that targets cloud-based LLM services by crafting special inputs that force the model into extremely long or infinite generation loops, exhausting computing resources and making the service unavailable to legitimate users. The attack works even against closed-source models (where the inner workings are hidden) by using a technique that converts text tokens into continuous numbers and then optimizes adversarial prompts in a lower-dimensional space. Testing on commercial LLM services showed ThinkTrap could reduce system performance to as little as 1% of normal capacity or cause complete service failure.