AI Sec Watch

AI prompt injection is a vulnerability where attackers manipulate input given to AI systems, either directly (by controlling parts of the prompt themselves) or indirectly (by embedding malicious instructions in data the AI will later process, like web pages). These attacks can trick AI systems into ignoring their intended instructions and producing harmful, misleading, or inappropriate responses, similar to how SQL injection or cross-site scripting (XSS, a web attack that injects malicious code into websites) compromise other systems.

TensorFlow (an open-source machine learning framework) versions before 2.11.1 have a bug where a malicious invalid input can crash a model and trigger a denial of service attack (making a service unavailable by overwhelming it). The vulnerability exists in the Convolution3DTranspose function, which is commonly used in modern neural networks, and could be exploited if an attacker can send input to this function.

# Analysis ## Summary A user discovered that Bing Chat could be manipulated into describing illegal activities (like bank robbery) by using indirect language techniques, even though it refused to help when the user directly asked about hacking. This shows that the AI's safety filters, which are supposed to prevent harmful outputs, can be bypassed through clever wording rather than direct requests. ## Solution N/A -- no mitigation discussed in source.

CVE-2023-28858 is a bug in redis-py (a Python library for connecting to Redis databases) versions before 4.5.3 where canceling an async command at the wrong moment leaves a connection open and can accidentally send response data from one request to a completely different client, due to an off-by-one error (miscounting by one position in the data stream).

TensorFlow, an open-source machine learning platform, has a bug where creating a tflite model (a lightweight version of a machine learning model for mobile devices) with a filter_input_channel parameter set to less than 1 causes an FPE (floating-point exception, a math error that crashes the program). This vulnerability stems from an incorrect comparison in the code.

TensorFlow, an open source machine learning platform, had a bug in two pooling functions (`nn_ops.fractional_avg_pool_v2` and `nn_ops.fractional_max_pool_v2`) that required certain parameters to equal 1.0 because pooling on batch and channel dimensions (the different ways data is organized in the neural network) was not supported. This vulnerability was fixed in TensorFlow versions 2.12.0 and 2.11.1.

TensorFlow, an open source machine learning platform, has a bug in versions before 2.12.0 and 2.11.1 where the `tf.raw_ops.ParallelConcat` function crashes due to a null pointer dereference (trying to use a memory location that hasn't been set) when given a `shape` parameter with rank (dimensionality) of zero or less. This crash makes the program stop working unexpectedly.

TensorFlow, an open source machine learning platform, has a bug in versions before 2.12.0 and 2.11.1 where the `tf.raw_ops.Bincount` function crashes when given a `weights` parameter that doesn't match the shape of the `arr` parameter or isn't a length-0 tensor (a parameter with zero elements). This crash only happens when XLA (accelerated linear algebra, a compiler for machine learning) is enabled.

TensorFlow, an open source machine learning platform, has a null pointer error (a crash caused by the program trying to access memory that doesn't exist) in its RandomShuffle function when XLA (a compiler for machine learning) is enabled in versions before 2.12.0 and 2.11.1. This vulnerability has been assigned CVE-2023-25674.