AI Sec Watch

This post describes how the author built Husky AI, a machine learning system that classifies images as huskies or non-huskies, using a convolutional neural network (CNN, a type of AI model designed to process images). The author gathered about 1,300 husky images and 3,000 other images using Bing Image Search, then organized them into separate training and validation folders to build and test the model. The post notes a potential security risk: attackers could poison either the training or validation image sets to cause the model to perform poorly.

This post introduces the machine learning pipeline, which consists of sequential steps from collecting training images, pre-processing data, defining and training a model, evaluating performance, and finally deploying it to production as an API (application programming interface, a way for software to communicate). The author uses a "Husky AI" example application that identifies whether uploaded images contain huskies, and explains that understanding this pipeline's components is important for identifying potential security attacks on machine learning systems.

A security researcher describes their year-long study of machine learning and AI fundamentals, with the goal of understanding how to build and then attack ML systems. The post outlines their learning approach, courses, and materials for others interested in starting adversarial machine learning (attacking ML systems).

Race conditions in ACL (access control list, the rules that determine who can access files) application occur when a system creates a sensitive file but there is a time gap before permissions are applied to protect it, potentially allowing attackers to access the file during that window. This type of vulnerability exploits the timing between file creation and permission lockdown to expose sensitive information.

Telemetry (data collected about how users interact with software) is often used by companies to make business decisions, but telemetry pipelines (the systems that collect and process this data) can be vulnerable to attacks. A red team security test demonstrated this by spoofing telemetry requests to falsely show a Commodore 64 as the most popular operating system, which could mislead companies into making poor decisions based on fake usage data.

This article discusses how to measure the maturity and effectiveness of security testing programs, particularly red teaming (simulated attacks to find vulnerabilities). The author suggests using existing frameworks like CMMI (Capability Maturity Model Integration, a system developed by Carnegie Mellon University that rates how well-organized software processes are on a scale of one to five) that can be adapted to evaluate offensive security programs.

This article discusses red teaming techniques (testing methods where security professionals act as attackers to find weaknesses) that organizations can use to identify privacy issues in their systems and infrastructure. The author emphasizes that privacy violations often come from insider threats (employees or contractors with authorized access to sensitive data), and highlights the importance of regular privacy testing as required by regulations like GDPR (General Data Protection Regulation, which sets rules for protecting personal data in Europe). The article mentions the "Motivated Intruder" threat model, where an insider with access to anonymized datasets (data with identifying information supposedly removed) uses data science techniques to reidentify people and expose their identities.

A vulnerability in Oracle Java SE's JAXP component (a tool for processing XML data) allows attackers to modify or delete data without authentication by sending malicious data through network protocols. The flaw affects multiple Java versions including 7u261, 8u251, 11.0.7, and 14.0.1, and has a CVSS score (a 0-10 rating of how severe a vulnerability is) of 5.3.

A researcher discovered a persistent XSS (cross-site scripting, where an attacker injects malicious code into a web page that runs in other users' browsers) vulnerability in the AWS Console several years ago. The post documents how they found the bug, the techniques they used, and Amazon's response to the discovery.