AI Sec Watch

Three major AI companies (OpenAI, Google, and Anthropic) submitted public comments to the U.S. government's request for input on developing an 'AI Action Plan' in response to President Trump's executive order. The companies largely advocated for increased government investment in AI infrastructure and public-private partnerships, though they framed their arguments differently, with OpenAI notably avoiding the term 'AI safety' in its response despite previous public emphasis on the topic.

A vulnerability in PyTorch 2.6.0+cu124 affects the torch.mkldnn_max_pool2d function, a component used for processing image data. The vulnerability can cause a denial of service (making a system unavailable), but requires local access to the machine. The vulnerability's real existence is still disputed.

Mesop is a Python-based UI framework for building web applications that has a class pollution vulnerability (a flaw allowing attackers to modify global variables and class attributes at runtime, similar to prototype pollution in JavaScript) in versions before 0.14.1. This vulnerability could cause denial of service attacks (making a service unavailable), identity confusion where attackers impersonate system roles, jailbreak attacks against LLMs (large language models, AI systems that generate text), or potentially remote code execution (running unauthorized commands on a server) depending on how the application is built.

OWASP (Open Worldwide Application Security Project, a nonprofit that helps organizations secure their software) has renamed and promoted its OWASP Top 10 for LLM (large language model, an AI trained on massive amounts of text data) project to the OWASP Gen AI Security Project, expanding its focus from just listing AI vulnerabilities to providing broader guidance on governance, risk management, and compliance for generative AI systems. The project now includes over 600 experts from 18 countries and has published new resources like the Agentic AI Threats and Mitigations Guide (addressing security risks in autonomous AI systems) along with translations in six additional languages.

In MLflow (a machine learning workflow tool) version 2.18, administrators can create user accounts without requiring passwords, which violates security best practices and could allow unauthorized access to accounts. This vulnerability is classified under weak password requirements, meaning the system doesn't enforce strong authentication measures.

A CSRF vulnerability (cross-site request forgery, where an attacker tricks a user into performing unwanted actions on a website) exists in the Signup feature of MLflow versions 2.17.0 to 2.20.1, allowing attackers to create unauthorized accounts. This could enable an attacker to perform malicious actions while appearing to be a legitimate user.

MLflow version 2.17.2 has a vulnerability in its `/graphql` endpoint (a web interface for querying data) that allows attackers to perform a denial of service attack (making a service unavailable) by sending large batches of repeated queries. This exhausts all the workers (processes handling requests) that MLflow has available, preventing the application from responding to legitimate requests.

Ollama (an AI model framework) versions 0.3.14 and earlier have a vulnerability where a malicious user can upload a specially crafted GGUF model file (a format for storing AI models) that causes a division by zero error (when code tries to divide a number by zero, crashing the program) in the ggufPadding function, crashing the server and making it unavailable (a Denial of Service attack).

A vulnerability in Ollama (an AI model software) version 0.3.14 and earlier allows an attacker to upload a specially crafted GGUF model file (a format for storing AI models) that tricks the server into using unlimited memory, causing a denial of service (DoS, a situation where a system becomes unavailable to users). The vulnerability stems from the server not properly limiting how much memory it allocates when processing model files.