AI Sec Watch

This research presents LipVor, an algorithm that mathematically verifies whether a trained neural network (a computer model with interconnected nodes that learns patterns) follows partial monotonicity constraints, which means outputs change predictably with certain inputs. The method works by testing the network at specific points and using mathematical properties to guarantee the network behaves correctly across its entire domain, potentially allowing neural networks to be used in critical applications like credit scoring where trustworthiness and predictable behavior are required.

text-generation-webui (an open-source tool for running large language models through a web interface) versions 3.13 and earlier contain a Local File Inclusion vulnerability (a flaw where an attacker can read files they shouldn't have access to) in the character picture upload feature. An attacker can upload a text file with a symbolic link (a shortcut to another file) pointing to sensitive files, and the application will expose those files' contents through the web, potentially revealing passwords and system settings.

This research addresses privacy risks in decentralized optimization (where multiple networked computers work together to solve a problem without a central coordinator) by proposing ZS-DDAPush, an algorithm that adds mathematical noise structures to protect sensitive node information during communication. The key innovation is that ZS-DDAPush achieves privacy protection while maintaining the accuracy and efficiency of the optimization process, avoiding the typical trade-offs seen in other privacy methods like differential privacy (adding statistical noise to protect individual data) or encryption (scrambling data so only authorized parties can read it).

Researchers developed TabExtractor, a tool that can steal tabular models (AI systems trained on spreadsheet-like data) without needing access to the original training data or knowing how the model was built. The attack works by creating synthetic data samples and using a special neural network architecture called a contrastive tabular transformer (CTT, a type of AI that learns by comparing similar and different examples) to reverse-engineer a clone of the victim model that performs almost as well as the original. This research shows that tabular models face serious security risks from extraction attacks.

Machine unlearning allows AI models to forget the effects of specific training samples, but verifying whether this actually happened is difficult because existing checks (like backdoor attacks or membership inference attacks, which test if a model remembers data by trying to extract or manipulate it) can be fooled by a dishonest model provider who simply retrains the model to pass the test rather than truly unlearning. This paper proposes IndirectVerify, a formal verification method that uses pairs of connected samples (trigger samples that are unlearned and reaction samples that should be affected by that unlearning) with intentional perturbations (small changes to training data) to create indirect evidence that unlearning actually occurred, making it harder to fake.

Researchers discovered a type of backdoor attack (hidden malicious instructions planted in AI systems) on multiagent reinforcement learning systems, where one adversary agent uses its actions to trigger hidden failures in other agents' decision-making policies. Unlike previous attacks that assumed unrealistic direct control over what victims observe, this attack is more practical because it works through normal agent interactions in partially observable environments (where agents cannot always see what others are doing). The researchers developed a training method to help adversary agents efficiently trigger these backdoors with minimal suspicious actions.

AI systems used for important decisions often rely on empirical risk minimization (ERM, a training method that reduces prediction errors on known data) to build models, but these systems can suffer from unintentional bias, lack of transparency, and other risks. The EU has established Ethics Guidelines requiring trustworthy AI to meet seven key requirements, yet current ERM-based design prioritizes accuracy over trustworthiness. This article argues that developers need to balance four core objectives when designing AI systems: fairness (not discriminating against groups), privacy (protecting user data), robustness (resisting intentional attacks like fake news), and explainability (being transparent about how decisions are made).

This research proposes a new method for deploying cyber deception (defensive tricks to confuse attackers) in networks by combining deep reinforcement learning (a type of AI that learns by trial and error) with game theory that accounts for time delays. The method uses an algorithm called proximal policy optimization (PPO, a technique for training AI to make optimal decisions) to figure out where and when to place deception resources, and tests show it outperforms existing approaches in handling complex network attacks.

This research presents a new method for generating counterfactual explanations (minimal changes needed to flip an AI model's prediction), which are a type of explainable AI that helps users understand why models make specific decisions. The approach combines physics concepts like energy minimization and simulated annealing (an optimization technique inspired by metallurgy) to find the smallest, most realistic modifications needed to change a model's output, with applications tested in cybersecurity for Internet of Things devices (networked physical devices like sensors and cameras).