AI Sec Watch

Hallucinations (instances where Large Language Models generate false or misleading content) are a safety problem for AI applications. The paper introduces UQLM, a Python package that uses uncertainty quantification (UQ, a statistical technique for measuring how confident a model is in its answer) to detect when an LLM is likely hallucinating by assigning confidence scores between 0 and 1 to responses.

This research paper studies diffusion models, a type of AI used to generate images and audio, as a statistical method for density estimation (learning the probability distribution of data). The authors show that when data has a factorizable structure (meaning it can be broken into independent low-dimensional components, like in Bayesian networks), diffusion models can efficiently learn this structure and achieve optimal performance using a specially designed sparse neural network architecture (one where most connections between neurons are inactive).

Auto-regressive video diffusion models (AR-VDMs, systems that generate videos by predicting one frame at a time) struggle with two problems: history forgetting, where they lose track of earlier frames they created, and temporal degradation, where video quality gets worse over time. Researchers created Meta-ARVDM, a theoretical framework that analyzes both problems and shows that using more past frames reduces history forgetting, while also introducing a new way to evaluate these models using a "needle-in-a-haystack" test (finding specific content buried in larger data).

This paper studies how to improve the accuracy of generative models (AI systems that create new data, like images or text) by using knowledge transfer, where a model trained on one task helps train a model on a different task. The researchers introduce a framework based on 'Shared Embedding,' a technique that finds common patterns between different tasks even when their data looks different, and show that this approach improves performance in two types of generative models: diffusion models (which gradually refine random noise into structured outputs) and normalizing flows (mathematical transformations that learn data distributions).

This paper presents a mathematical framework for understanding denoising Markov models (generative models that learn to reverse a noising process to create new data). The authors use concepts from statistical mechanics to establish rigorous rules for how these models work, unifying existing approaches like diffusion models and proposing new variations using different types of mathematical processes.

This research addresses how stochastic convex optimization (a machine learning technique for finding the best solution by processing data in random batches) can work when key problem parameters are unknown. The authors propose two methods: a model selection technique that prevents overfitting (when an AI learns noise in the validation data instead of real patterns), and a regularization-based approach that estimates unknown parameters to achieve optimal efficiency. Experiments on image classification and shape-counting tasks show these methods help reduce overfitting on small validation sets.

This research introduces Abstract Gradient Training (AGT), a framework for verifying that machine learning models remain reliable when their training data is changed or corrupted. The framework addresses three scenarios: adversarial data poisoning (when attackers intentionally alter training samples), machine unlearning (when specific training data must be removed), and differential privacy (when individual data points are substituted). AGT works by establishing mathematical bounds on how model parameters (the internal settings that AI uses to make predictions) can change, allowing researchers to formally prove a model will behave safely despite training data perturbations.

This research addresses problems with training implicit generative models (AI systems that learn to create new data similar to real data) by proposing the invariant statistical loss (ISL), which avoids unstable adversarial training by comparing the statistical ranks of real and generated samples. The authors improve ISL for two practical scenarios: using a Pareto distribution instead of Gaussian noise to better model extreme values in data, and introducing ISL-slicing to handle large multivariate datasets (data with many variables) by projecting onto random lower-dimensional subspaces.

This research paper analyzes denoising reflected diffusion models, which are a type of generative AI (systems that create new data like images or text). The study shows that reflected diffusion processes (a mathematical technique using boundaries to keep the model's state space bounded) can match theoretical predictions better than standard diffusion models, and provides mathematical proof of how quickly these models converge to accurate results.