AI Sec Watch

A bug in TensorFlow's tf.raw_ops.ImmutableConst operation (a function that creates fixed tensors from memory-mapped files) causes the Python interpreter to crash when the tensor type is not an integer type, because the code tries to write to memory that should be read-only. This crash happens when the file is large enough to contain the tensor data, resulting in a segmentation fault (a critical memory access error).

CVE-2020-26267 is a vulnerability in TensorFlow where the tf.raw_ops.DataFormatVecPermute API (a function for converting data format layout) fails to check the src_format and dst_format inputs, leading to uninitialized memory accesses (using memory that hasn't been set to a known value), out-of-bounds reads (accessing data outside intended boundaries), and potential crashes. The vulnerability was patched across multiple TensorFlow versions.

CVE-2020-26266 is a vulnerability in TensorFlow where saved models can accidentally use uninitialized values (memory locations that haven't been set to a starting value) during execution because certain floating point data types weren't properly initialized in the Eigen library (a math processing component). This is a use of uninitialized resource (CWE-908) type bug that could lead to unpredictable behavior when running affected models.

TensorFlow has a vulnerability where loading a saved model can access uninitialized memory (data that hasn't been set to a known value) when building a computation graph. The bug occurs in the MakeEdge function, which connects parts of a neural network together, because it doesn't verify that array indices are valid before accessing them, potentially allowing attackers to leak memory addresses from the library.

A bug was found in the Linux kernel before version 5.7.3 in the get_user_pages function (a mechanism that allows programs to access memory pages), where it incorrectly grants write access when it should only allow read access for copy-on-write pages (memory regions shared between processes that are copied when modified). This happens because the function doesn't properly respect read-only restrictions, creating a security vulnerability.

This is an index page summarizing a series of blog posts about machine learning security from a red teaming perspective (testing a system by simulating attacker behavior). The posts cover ML basics, threat modeling, practical attacks like adversarial examples (inputs designed to fool AI models), model theft, backdoors (hidden malicious code inserted into models), and how traditional security attacks (like weak access control) also threaten AI systems.

This post describes how Generative Adversarial Networks (GANs, a type of AI system where two neural networks compete to create realistic fake images) can be used to generate fake husky photos that trick an image recognition system called Husky AI into misclassifying them as real huskies. The author explains they investigated this attack method and references a GAN course to learn more about the technique.

AI and machine learning systems have caused serious problems in real-world situations, including Amazon's recruiting tool that discriminated against women, Microsoft's chatbot that became racist and sexist, IBM's cancer treatment recommendation system that doctors criticized, and Facebook's AI that made incorrect translations leading to someone's arrest. These examples show that AI systems can develop and spread biased predictions and failures with harmful consequences. The article highlights the importance of addressing bias when building and deploying AI systems responsibly.

A vulnerability in Libsvm v324 (a machine learning library used by scikit-learn 0.23.2) allows attackers to crash a program by sending a specially crafted machine learning model with an extremely large value in the _n_support array, causing a segmentation fault (a type of crash where the program tries to access memory it shouldn't). The scikit-learn developers noted this only happens if an application violates the library's API by modifying private attributes.