AI Sec Watch

MLflow versions before v3.7.0 contain a command injection vulnerability (a flaw where attackers insert malicious commands into input that gets executed) in the sagemaker module. An attacker can exploit this by passing a malicious container image name through the `--container` parameter, which the software unsafely inserts into shell commands and runs, allowing arbitrary command execution on affected systems.

This week's security news includes Google patching two actively exploited Chrome vulnerabilities in the graphics and JavaScript engines that could allow code execution, Meta discontinuing encrypted messaging on Instagram, and law enforcement disrupting botnets (malware networks that hijack routers) like SocksEscort and KadNap that were being used for fraud and illegal proxy services. A threat actor also exploited a compromised npm package (a JavaScript code library) to breach an AWS cloud environment and steal data.

Shadow AI refers to AI tools used throughout an organization without IT oversight or approval, creating security and governance challenges. The source describes Nudge Security as a platform that addresses this by providing continuous discovery of AI apps and user accounts, monitoring for sensitive data sharing in AI conversations, and tracking which AI tools have access to company data through integrations.

This article examines how large language models (AI systems trained on huge amounts of text data) can be used in cybersecurity red teaming (simulated attacks to test defenses) and blue teaming (defensive security work), mapping their abilities to established security frameworks. However, LLMs struggle in difficult, real-world situations because they have limitations like hallucinations (generating false information confidently), poor memory of long conversations, and gaps in logical reasoning.

Autonomous AI agents (AI systems that operate independently to complete complex tasks with minimal human oversight) have advanced rapidly, creating new governance challenges because they can operate at machine speed without humans in the loop to approve each decision. Unlike traditional chatbots where humans reviewed outputs before consequential actions, agents now directly modify enterprise systems and data, making organizations legally liable for any harm caused (similar to how parents are responsible for their children's actions). Without building governance rules directly into the code that controls these agents' permissions and actions, organizations face significant risks from drift (where agents behave differently than intended) and unauthorized access to critical systems.

Organizations typically use separate security tools (BAS tools, pentesting products, vulnerability scanners) that don't communicate with each other, creating blind spots because attackers chain multiple vulnerabilities together in coordinated operations. The article proposes that agentic AI (autonomous AI agents that can plan, execute, and reason through complex tasks without human direction at each step) should be applied to security validation to create a unified, continuous system that combines adversarial perspective (how attackers get in), defensive perspective (whether defenses stop them), and risk perspective (which exposures actually matter).

Threat actors are spreading GlassWorm malware through Open VSX extensions (add-ons for the VS Code editor) by abusing dependency relationships, a feature that automatically installs other extensions when one is installed. Instead of hiding malware in every extension, attackers create legitimate-looking extensions that gain user trust, then update them to depend on separate extensions containing the malware loader, making the attack harder to detect.

OpenAI is developing an "adult mode" for ChatGPT that will allow users to generate text conversations with adult themes, described as "smut" rather than pornography. The feature will initially support only text and will not generate images, voice, or video content. OpenAI claims to have reduced "serious mental health issues" in its AI model enough to safely relax safety restrictions (the guardrails that prevent the AI from producing certain types of content) for this feature.

OWASP, a nonprofit cybersecurity organization, has published a checklist to help companies secure their use of generative AI and LLMs (large language models, which are AI systems trained on massive amounts of text to understand and generate human language). The checklist covers six key areas: understanding competitive and adversarial risks, threat modeling (identifying how attackers might exploit AI systems), maintaining an inventory of AI tools and assets, and ensuring proper governance and security controls are in place.