AI Sec Watch

TensorFlow (an open source machine learning platform) has a vulnerability where tensors (multi-dimensional arrays of numbers) with very large dimensions can cause an integer overflow (when a calculation produces a number too big to store), resulting in a crash or inconsistent behavior. The vulnerability occurs because the code checks for overflow incorrectly in some parts of the codebase.

TensorFlow (an open source machine learning platform) has a bug in its Keras pooling layers (functions that reduce data size by sampling from groups of values) that can cause a segfault (crash where the program tries to access invalid memory) if the pool size is 0 or if a dimension is negative, because the code doesn't check that these values are positive.

TensorFlow's `tf.math.segment_*` operations (functions that process data divided into segments) crash with a denial of service error when a segment ID is very large, because the code doesn't properly handle cases where the output size exceeds what an int64_t (a 64-bit integer type) can store. The crash happens in both CPU and GPU implementations when computing output shape.

CVE-2021-42694 is a vulnerability in the Unicode Specification (up to version 14.0) that allows attackers to create source code identifiers (like function names) using homoglyphs (characters that look identical but are technically different) to sneak malicious code into software. An attacker could use these visually identical but distinct characters in upstream dependencies (external code libraries), and developers reviewing the code might not catch the deception, allowing the malicious code to be used downstream (in other software that depends on it).

Rasa is a framework for building conversational AI systems, and versions before 2.8.10 have a vulnerability where a malicious model file (a compressed archive containing trained AI weights) can overwrite or replace important bot files. This happens because the software doesn't properly validate what's inside the model file before extracting it.

Adversaries can hide a smaller image within a larger one so that it becomes visible when a computer resizes the image using insecure interpolation (a method of calculating pixel values between known points). The video demonstrates this attack technique and explains how to prevent it from happening.

ParlAI, a framework for training AI models on dialogue datasets, has a vulnerability where it unsafely loads YAML files (a data format), allowing attackers to execute arbitrary code on affected systems. The vulnerability occurs because the framework uses an unsafe YAML loader that can be tricked into running malicious code hidden in data files.

This post describes Microsoft Counterfit, a tool for testing machine learning models against adversarial attacks (subtle modifications to input data designed to fool AI systems). The author demonstrates how to set up Counterfit, create a custom target for a husky image classifier, and use the tool's built-in attack modules to test the model's robustness.

TensorFlow, an open-source machine learning platform, had a bug where certain shape functions created temporary data structures (ShapeAndType structs) that were deleted too quickly, causing crashes (segfaults, or sudden program failures) if other code tried to access them. The issue was that while normal output shapes were being protected by copying them to safer ownership, the code wasn't doing the same protection for shapes and types together.