AI Sec Watch

Industrial process monitoring systems often perform poorly when dealing with noise (unwanted signal disturbances) and outliers (unusual data points) across different working conditions. This research proposes a transfer learning (a technique where a model trained on one task is adapted for a different but related task) framework that combines several advanced neural network approaches, including a variational autoencoder generative adversarial network (a type of AI that learns to generate and discriminate realistic data patterns) and dictionary learning (a method that finds the simplest way to represent complex data), to make monitoring systems more robust and reliable across different industrial scenarios.

The article demonstrates how attackers can use crafted prompts to trick AI assistants into running harmful database queries through prompt-to-SQL injection attacks (where malicious instructions hidden in user input cause an AI to generate dangerous database commands). It identifies vulnerabilities in real systems and describes practical defenses including query filtering, rewriting, data preloading, and using another AI model as a security guard.

NVIDIA Merlin Transformers4Rec contains a code injection vulnerability (CWE-94, a weakness where attackers can trick software into running malicious code) that could let attackers execute arbitrary code, gain elevated permissions, steal information, or modify data. The vulnerability affects all platforms running this software. A CVSS severity score has not yet been assigned by NIST.

This research paper analyzes how companies that invest in digital technologies, including AI, affect their greenhouse gas emissions and natural resource use. The study found that companies investing in these technologies tend to reduce their emissions and consume fewer natural resources, suggesting that digital tools can help address environmental challenges.

ChatterBot versions up to 1.2.10 have a vulnerability that causes denial-of-service (when a service becomes unavailable due to being overwhelmed), triggered when multiple concurrent calls to the get_response() method exhaust the SQLAlchemy connection pool (a group of reusable database connections). The service becomes unavailable and requires manual restart to recover.

Researchers studied how multimodal generative AI systems (AI that creates both text and images) could be misused to generate fake news by testing AI image generators like Stable Diffusion 3 and evaluating whether GPT-4 (a large language model) could detect these false images. The findings show that current AI has both strengths and weaknesses in creating misinformation, and highlight that stronger safeguards, transparent monitoring, and human oversight are needed to prevent AI from spreading false information.

BlindU is a method that allows users to remove their data's influence from trained AI models while keeping that data hidden from the server. Instead of uploading raw data to the server (which creates privacy risks), BlindU lets users create compressed versions of their data locally, and the server performs the removal process only on these compressed versions, making it practical for federated learning (a distributed training setup where data stays on users' devices).

This research addresses how to remove unwanted information from pre-trained vision models (AI systems trained to understand images) when users or model owners request it, especially when these deletion requests come one after another over time. The researchers propose Group Sparse LoRA (GS-LoRA), a technique that uses Low-Rank Adaptation modules (efficient add-on components that modify specific neural network layers) to selectively forget targeted classes or information while keeping the rest of the model working well, even when some training data is missing.

Cursor is a code editor designed for programming with AI. Before version 2.3, when the Cursor Agent runs in Auto-Run Mode with Allowlist mode enabled (a security setting that restricts which commands can run), attackers could bypass this protection by using prompt injection (tricking the AI by hiding instructions in its input) to execute shell built-ins (basic operating system commands) and modify environment variables (settings that affect how programs behave). This vulnerability allows attackers to compromise the shell environment without user approval.