AI Sec Watch

ChromaDB (a Python project for storing AI embeddings) versions 1.0.0 and later contain a code injection vulnerability that lets unauthenticated attackers run arbitrary code on the server by sending a malicious model repository with a specific setting enabled to a particular API endpoint. The vulnerability has a CVSS score (a 0-10 severity rating) of 10.0, marking it as critical.

AI models are rapidly improving at performing multi-stage penetration tests (simulated attacks where attackers try to break into systems through multiple steps), with the difficulty of tasks they can complete doubling every 4.7 months as of early 2025. The UK government's AI Security Institute measured this by comparing how well AI models could replicate tasks that human cybersecurity experts can complete, finding that the latest AI systems are now showing even greater capability and posing real security risks to organizations with weak defenses.

Amazon has added a new feature to Alexa Plus (its upgraded AI assistant) that lets users generate podcasts on nearly any topic by simply describing what they want. The AI creates two AI-generated hosts that discuss the chosen topic, and users can preview and customize the podcast before it's created.

Anthropic is briefing global financial regulators on Claude Mythos, an AI model with advanced capabilities in finding previously unknown cybersecurity flaws that hackers could exploit. Rather than releasing Mythos publicly, Anthropic has restricted access to selected tech companies and banks to help identify weaknesses, while the UK's AI Security Institute has found the latest version shows a significant capability jump, even completing a difficult autonomous hacking test that no prior model had solved.

Penetration testing (PT, a security technique where experts simulate attacks to find weaknesses in networks) is traditionally slow and expensive because it relies heavily on manual expert work. This research proposes an automated and intelligent PT method using a knowledge graph (a structured database of relationships between network components) and historical decision data to create realistic test environments and make smarter attack decisions, achieving a 69% reduction in repeated testing attempts.

This paper presents CyberOps-Bots, a system that combines Large Language Models (LLMs, which are AI models trained on text) with reinforcement learning (RL, a type of AI that learns by trial and error) to defend cloud networks against attacks. The system uses a two-layer approach where an upper LLM agent handles planning and human input, while lower RL agents execute specific defense actions, and testing shows it maintains network availability much better than existing methods without needing to retrain when network conditions change.

ROSE is a black-box watermarking method (a technique to prove ownership of AI models by embedding hidden triggers that only the owner can activate) for protecting deep neural networks (DNNs, large AI models that learn patterns from data) in machine learning services. The method uses secret trigger-label pairs connected through a hash function to verify ownership while resisting attacks like fine-tuning, pruning, and other model modifications, while maintaining the model's performance on its original task.

Connected autonomous vehicles share sensor data to improve driving safety, but this collaboration is vulnerable to adversarial message attacks (malicious input designed to fool AI systems) from bad actors that can degrade performance. The paper describes a new blind area confusion attack that bypasses existing defenses, then proposes GCP, a framework that detects malicious agents by checking both spatial consistency (whether sensor readings from different vehicles agree) and temporal anomalies (unusual patterns over time) using statistical testing methods.

This research presents a new AI framework for network intrusion detection systems (IDS, which are tools that identify unauthorized access attempts on computer networks) that provides both accurate threat detection and reliable confidence levels in its predictions. The framework combines transformer models (a type of neural network architecture) with ensemble methods (combining multiple AI models for better results) to break down prediction uncertainty into two types: epistemic uncertainty (uncertainty from the model itself) and aleatoric uncertainty (uncertainty from noisy or incomplete data). Testing on four benchmark datasets shows the system achieves strong detection rates (77.55% to 97.00% F1-scores, a measure of accuracy) while maintaining good calibration (accurate confidence estimates) and remaining resilient to adversarial attacks (attempts to fool the AI with specially crafted malicious inputs).