AI Sec Watch

TensorFlow (an open source machine learning platform) has a bug in its Keras pooling layers (functions that reduce data size by sampling from groups of values) that can cause a segfault (crash where the program tries to access invalid memory) if the pool size is 0 or if a dimension is negative, because the code doesn't check that these values are positive.

TensorFlow's `tf.math.segment_*` operations (functions that process data divided into segments) crash with a denial of service error when a segment ID is very large, because the code doesn't properly handle cases where the output size exceeds what an int64_t (a 64-bit integer type) can store. The crash happens in both CPU and GPU implementations when computing output shape.

CVE-2021-42694 is a vulnerability in the Unicode Specification (up to version 14.0) that allows attackers to create source code identifiers (like function names) using homoglyphs (characters that look identical but are technically different) to sneak malicious code into software. An attacker could use these visually identical but distinct characters in upstream dependencies (external code libraries), and developers reviewing the code might not catch the deception, allowing the malicious code to be used downstream (in other software that depends on it).

Rasa is a framework for building conversational AI systems, and versions before 2.8.10 have a vulnerability where a malicious model file (a compressed archive containing trained AI weights) can overwrite or replace important bot files. This happens because the software doesn't properly validate what's inside the model file before extracting it.

Adversaries can hide a smaller image within a larger one so that it becomes visible when a computer resizes the image using insecure interpolation (a method of calculating pixel values between known points). The video demonstrates this attack technique and explains how to prevent it from happening.

ParlAI, a framework for training AI models on dialogue datasets, has a vulnerability where it unsafely loads YAML files (a data format), allowing attackers to execute arbitrary code on affected systems. The vulnerability occurs because the framework uses an unsafe YAML loader that can be tricked into running malicious code hidden in data files.

This post describes Microsoft Counterfit, a tool for testing machine learning models against adversarial attacks (subtle modifications to input data designed to fool AI systems). The author demonstrates how to set up Counterfit, create a custom target for a husky image classifier, and use the tool's built-in attack modules to test the model's robustness.

TensorFlow, an open-source machine learning platform, had a bug where certain shape functions created temporary data structures (ShapeAndType structs) that were deleted too quickly, causing crashes (segfaults, or sudden program failures) if other code tried to access them. The issue was that while normal output shapes were being protected by copying them to safer ownership, the code wasn't doing the same protection for shapes and types together.

TensorFlow (an open source machine learning platform) had a bug where Go code could crash the program during memory cleanup of string tensors if encoding failed. The problem occurred because the cleanup process assumed encoding always succeeded, but didn't check whether it actually did.