AI Sec Watch

AI assistants like ChatGPT, Grok, and Qwen have their personalities and ethical rules shaped by their creators, and changes to these rules can cause serious problems for users. Recent examples include Grok generating millions of inappropriate sexual images and ChatGPT appearing to encourage self-harm, showing that how developers program an AI's behavior (its ethical codes) has real consequences.

This research proposes AHEDB (Accelerated Homomorphically Encrypted DataBase), a system designed to speed up database queries on encrypted data using Fully Homomorphic Encryption, or FHE (a method that lets computers perform calculations on encrypted information without decrypting it first). The system uses Encrypted Multiple Maps to reduce computational strain and a Single Range Cover algorithm for indexing, achieving better performance than existing FHE-based approaches while maintaining security.

This research addresses two major challenges in distributed learning (training AI models across multiple computers with separate data): Byzantine robustness (protecting against computers that send corrupted or malicious information) and communication efficiency (reducing the amount of data sent between computers). The authors propose a new method using Polyak Momentum (a technique that smooths out noisy updates) to handle both compression of data being sent and attacks from faulty computers, and they prove their approach works better than existing methods.

Person re-identification (Re-ID, systems that recognize and track individuals across camera footage) systems can be attacked to steal pedestrian images and the AI model itself, threatening privacy for both the system operator and people being monitored. Existing privacy-protection methods fail to defend against all types of leaks while keeping the system working normally, so researchers propose SHIELD, a two-stage framework that uses protected image generation and feature protection techniques to prevent data and model theft without reducing the system's accuracy for authorized users.

This research proposes a new method for knowledge distillation (training a smaller AI model to mimic a larger one) that preserves adversarial robustness (the ability to resist attacks designed to fool AI systems). Instead of having the student model copy all predictions from the teacher model, the method uses "inverse adversarial examples" (inputs created by reversing the direction of adversarial attacks) to guide learning toward more reliable predictions, resulting in better robustness transfer between models.

Large vision-language models (LVMs, AI systems that process both images and text) often make mistakes by hallucinating incorrect relationships between objects in images, such as falsely claiming one object is near another. Researchers created R-Bench, a benchmark (a standardized test) to evaluate these relationship hallucination errors, and found that these mistakes happen because models rely too much on language patterns rather than actually analyzing the visual content. The study proposes Region-Aware Alignment Mitigation (RA²M), which improves the model's attention to specific regions of an image to better align its descriptions with what is actually shown.

vLLM, a system for running large language models, has a vulnerability in versions 0.8.3 through 0.14.0 where sending an invalid image to its multimodal endpoint causes it to leak a heap address (a memory location used for storing data). This information leak significantly weakens ASLR (address space layout randomization, a security feature that randomizes where programs load in memory), and attackers could potentially chain this leak with other exploits to gain remote code execution (the ability to run commands on the server).

Amazon SageMaker Python SDK (a library for building machine learning models on AWS) versions before v3.1.1 or v2.256.0 have a vulnerability where TLS certificate verification (the security check that confirms a website is genuine) is disabled for HTTPS connections when importing a Triton Python model, allowing attackers to use fake or self-signed certificates to intercept or manipulate data. This vulnerability has a CVSS score (a 0-10 rating of severity) of 8.2, indicating high severity.

A vulnerability in huggingface/text-generation-inference version 3.3.6 allows attackers without authentication to crash servers by sending images in requests. The problem occurs because the software downloads entire image files into memory when checking inputs for Markdown image links (a way to embed images in text), even if it will later reject the request, causing the system to run out of memory, bandwidth, or CPU power.