New tools, products, platforms, funding rounds, and company developments in AI security.
This is a podcast episode about AI red teaming (simulated attacks to find weaknesses in AI systems) and threat modeling (planning for potential security risks) in machine learning systems. The episode explores how traditional security practices can be combined with machine learning security to better protect AI applications from attacks.
Yolo is a tool that uses ChatGPT API (OpenAI's language model accessed through code) to translate natural language questions into shell commands (the text-based interface for controlling a computer) that can be executed automatically. The tool helps users who forget command syntax by converting plain English requests into proper bash, zsh, or PowerShell commands, with a safety feature that shows the command before running it unless the user enables automatic execution.
This post announces a video tutorial about SSH Agent Hijacking, a technique (TTP, or tactic/technique/procedure) used in security testing where an attacker compromises the SSH Agent (a program that stores SSH keys, which authenticate users to remote systems). The tutorial is intended to help security professionals understand this attack method and develop ways to detect it on Linux and macOS systems.
This article describes how to capture and decrypt TLS traffic (the encryption protocol that secures web communication) on Windows using built-in tools like netsh and Wireshark. The technique works by setting an environment variable to log TLS session keys, capturing network traffic without installing extra software, and then using Wireshark to decrypt the captured data.
The EU AI Act requires technical standards to be written by European standardization organizations (CEN and CENELEC) that explain how companies can safely build high-risk AI systems. These standards follow a six-step approval process and, once published and approved by the European Commission, become 'harmonized and cited standards' that legally presume compliance with safety regulations if companies follow them. The drafting process is currently ongoing but behind schedule, with different standards at different completion stages.
Attackers are using a new phishing technique that exploits OAuth2 Device Authorization Grant (a protocol that lets devices log in by directing users to a website to authorize access) to trick users into granting them access to data, especially as organizations adopt passwordless authentication methods like hardware tokens. The attack begins when an attacker initiates a device code flow by sending a request to a service provider's device code endpoint, such as Microsoft's Azure AD (active directory, a system that manages user identities and permissions).
The article discusses security risks with Azure's AAD (Azure Active Directory, Microsoft's cloud identity service) when MFA (multi-factor authentication, an extra security check beyond passwords) is misconfigured. A common mistake is enforcing MFA only at the federated identity provider (an external service that handles logins) while leaving ROPC (Resource Owner Password Credentials, a method where users send passwords directly to get access tokens) enabled in AAD itself, which allows attackers to bypass MFA protections after gaining initial access.
PenTest Magazine's latest free issue features an article about ropci, a tool used in penetration testing (simulated security attacks to find vulnerabilities). The article provides more detailed information than the tool's blog post or GitHub repository, and the magazine also includes coverage of Nuclei, another popular security testing tool.
ROPC (Resource Owner Password Credentials, an OAuth2 method that sends usernames and passwords directly to get access tokens) can bypass multi-factor authentication (MFA, a security check requiring multiple forms of verification) in Microsoft Azure Active Directory if not properly configured, because over 50 default apps in every tenant support ROPC and may not enforce MFA. The post warns that ROPC should not be used and recommends testing your Azure setup for this vulnerability using the ropci tool.
SSH Agent Hijacking is an attack where an adversary with root permissions can steal SSH private keys (encryption keys used for secure shell access) from a forwarded SSH Agent, a service that stores authentication credentials. On shared systems like jumpboxes (intermediate servers used to access other machines), an attacker can find another user's SSH_AUTH_SOCK (an environment variable pointing to the SSH Agent's communication socket) and use it to impersonate that user and access machines they have permission to reach.
Gospray is a tool designed to test credentials by attempting to log into domain controllers (the central servers that manage network access) using LDAP bind (a method for authenticating directly against these servers). Unlike some other credential-testing approaches, it communicates quietly and directly with the domain controller without needing other servers as intermediaries, and it allows configurable levels of parallel attempts.
Running pip download (a Python command that downloads packages without installing them) can execute malicious code on your computer due to a design flaw, even though many people assume only pip install poses a security risk. This vulnerability allows attackers to run arbitrary code (commands of their choice) simply by downloading a compromised package.
This post demonstrates how attackers can use eBPF (extended Berkeley Packet Filter, a technology that lets programs run safely inside the Linux kernel) and bpftrace (a tool for tracing system events) to intercept and steal PAM (Pluggable Authentication Modules, the system that handles login authentication) passwords when users log in. The technique works similarly to a backdoor PAM module but uses eBPF instead, highlighting emerging security risks from new kernel-level attack tools.
This post describes a post-exploitation attack where an attacker modifies PAM (Pluggable Authentication Modules, a Unix system for controlling login behavior) to secretly capture user passwords by inserting a malicious script into the authentication pipeline. The attacker creates a bash script that logs usernames, passwords, and IP addresses whenever someone logs in, making it difficult to detect because it operates within legitimate system authentication mechanisms.
This article explains how to customize shell prompts (the command-line interface where users type commands) to automatically display date and time information, which helps security professionals document their activities during red team exercises (simulated attacks to test security). The article provides examples for three common shells: Bash, zsh on macOS, and PowerShell.
LLM outputs are untrusted and can be manipulated through prompt injection (tricking an AI by hiding instructions in its input), which affects large language models in particular ways. This post addresses how to handle the risks of untrusted output when using AI systems in real applications.
AI prompt injection is a vulnerability where attackers manipulate input given to AI systems, either directly (by controlling parts of the prompt themselves) or indirectly (by embedding malicious instructions in data the AI will later process, like web pages). These attacks can trick AI systems into ignoring their intended instructions and producing harmful, misleading, or inappropriate responses, similar to how SQL injection or cross-site scripting (XSS, a web attack that injects malicious code into websites) compromise other systems.
# Analysis ## Summary A user discovered that Bing Chat could be manipulated into describing illegal activities (like bank robbery) by using indirect language techniques, even though it refused to help when the user directly asked about hacking. This shows that the AI's safety filters, which are supposed to prevent harmful outputs, can be bypassed through clever wording rather than direct requests. ## Solution N/A -- no mitigation discussed in source.
This post demonstrates that ChatGPT can be prompted to roleplay as a Microsoft SQL Server (a database management system) and respond with realistic database commands and results, including creating databases, tables, inserting data, and writing stored procedures (reusable blocks of SQL code). The author shows that ChatGPT can understand user intent well enough to execute complex database operations like UPSERTs (operations that update existing records or insert new ones if they don't exist), even when given incomplete information.
Fix: The source mentions testing as a mitigation: 'Always enforce MFA' and 'test your own AAD tenant for ROPC based MFA bypass opportunities.' Additionally, the post states 'ROPC MUST NOT be used' as a general security principle. However, no specific configuration steps, patches, or enforcement mechanisms are explicitly described in the provided text.
Embrace The RedPickle files (Python's serialization format for saving objects) can be backdoored because they execute code through opcodes (instructions that control a virtual machine). Attackers can inject malicious commands into pickle files using tools like fickling, and when someone loads the file, the hidden code runs without interrupting the program's normal function. This is especially dangerous in shared environments like Google Colab, where an infected pickle file could give attackers access to a user's connected Google Drive.
Fix: The source mentions fickling, a tool by Trail of Bits that can both inject code into pickle files and check them for backdoors using two built-in safety features: '--check-safety' (which checks for malicious opcodes) and '--trace' (which shows the various opcodes). The source also recommends: "only ever open pickle files that you created or trust."
Embrace The RedFix: The source explicitly mentions three mitigation strategies: (1) 'Test EDR to catch modifications in PAM configurations (also binary patching or entirely replacing/backdooring existing ones)', (2) 'Review the PAM modules and there configuration in your environments', and (3) 'Do a purple team exercise that focuses on PAM modules and related configuration files'.
Embrace The Red