AI Sec Watch

LibreChat is a ChatGPT-like application that works with multiple AI providers. Before version 0.8.4-rc1, a file upload endpoint called POST /api/convos/import didn't have proper file size restrictions, allowing logged-in users to upload very large files that could fill up a server's storage and memory. A previous security fix added size limits to other file uploads but missed this endpoint.

A German court ruled that Google is liable for inaccurate AI search summaries, rejecting the argument that Google is merely a neutral carrier of information. The ruling clarifies that when companies use AI to rewrite and summarize content from other sources, they act as publishers and editors, making editorial decisions like traditional newspapers, rather than simply transmitting information. This legal development suggests that companies deploying AI systems bear responsibility for their accuracy, similar to how they would be liable if human employees made the same errors.

Claude Code's `/copy` command had a serious security flaw where it saved responses to an easily guessable file location (`/tmp/claude/response.md`) that any user on the system could read, potentially exposing secrets or credentials. An attacker could also create a symlink (a shortcut to another file) at that location to trick the command into overwriting any file they chose. This vulnerability required the attacker and a privileged user to be on the same computer.

A macOS malware called "Gaslight" uses prompt injection (tricking an AI by hiding instructions in its input) to confuse AI-powered malware analysis tools by embedding fake error messages, crash reports, and debugging data within the executable file. The malware contains 38 fabricated system messages designed to make LLM (large language model)-assisted analysis tools question their own sessions or stop analyzing the malware, rather than trying to evade detection in sandboxes (isolated test environments). Researchers attribute the malware to a North Korean-linked threat actor, and while it hasn't been shown to successfully bypass current AI analysis platforms, it suggests attackers are developing new anti-analysis techniques targeting AI-based security tools.

LibreChat, a ChatGPT-like application supporting multiple AI providers, has a security flaw in versions before 0.8.4-rc1 where an attacker with a valid session token (a code that proves you're logged in) can disable a user's two-factor authentication (2FA, an extra security layer requiring a second verification step) without permission. The attacker can overwrite the TOTP secret (a code used to generate login verification codes) and backup codes, then disable 2FA entirely, locking the real owner out of their account.

This research presents a new protocol called Updatable MPSI (UMPSI) that helps multiple organizations securely share threat intelligence data while keeping their information private. Unlike existing systems that require expensive full re-computation when threat data changes, UMPSI uses lightweight mechanisms to quickly update only the new or changed threat indicators, making real-time collaborative threat detection practical.

Representation engineering (RepE, a technique that modifies how an AI model processes information internally) defenses against jailbreaks (methods that trick LLMs into ignoring safety rules) have relied on block-list approaches that learn from known harmful examples, but this leaves gaps against new attacks. The paper proposes Ellipsoid Control, a white-list defense that instead protects the model's normal, harmless behavior by using test-time defense (constraints applied when the model runs) with projected gradient descent (a mathematical technique that adjusts the model's internal state while keeping it close to safe boundaries) to trigger refusals on harmful inputs while preserving the model's ability to answer legitimate questions.

This research presents a new algorithm for training predictive models on sensitive data while protecting privacy using differential privacy (DP, a mathematical technique that adds noise to data to prevent identifying individuals). The algorithm extends previous privacy-preserving methods to handle weighted empirical risk minimization (wERM, where different data points contribute differently to model training), which is particularly useful for personalized medical treatment decisions. Testing shows the approach successfully protects privacy while keeping the trained models effective.

This research addresses two problems in federated learning (a training approach where multiple computers work together to improve a shared AI model without sending raw data to one central location): client drift (when different computers' local updates diverge from each other due to unequal data) and Byzantine attacks (when some computers send false or malicious updates to sabotage the model). The paper proposes two frameworks called DRAG and BR-DRAG that use a metric called divergence of degree to measure how far off-track each computer's updates are, then automatically correct them through a process called linear calibration, with BR-DRAG adding a vetted dataset at the main server to defend against malicious actors.