AI Sec Watch

AI agents (software programs that can perform tasks automatically) are being promoted as the next major breakthrough, but companies are discovering they are unreliable and expensive to operate. The main problems include wasting tokens (units of text that AI processes, which cost money), high inference costs (the expense of running AI models), and system complexity that makes it difficult to manage multiple agents working together without burning through budgets instead of saving money.

Anthropic released Claude Opus 4.7 in April 2026 with notable updates to its system prompt (the hidden instructions that guide how an AI behaves), including expanded child safety rules, new tools like Claude in PowerPoint and Chrome browsing agents, and changes to make the model less verbose and more action-oriented. The update shows Anthropic shifting Claude toward trying to solve ambiguous requests using available tools rather than asking users for clarification first.

A researcher converted Anthropic's published Claude system prompts (the hidden instructions that guide Claude's behavior) from a single markdown document into a git repository (a version control system that tracks file changes over time) with timestamped commits, allowing easier exploration of how the prompts have evolved across different Claude model versions using standard git tools like `log` and `diff`.

This is a research survey published in ACM Computing Surveys that examines the limitations and problems of large language models (LLMs, which are AI systems trained on massive amounts of text data to generate human-like responses). The survey takes a data-driven approach to understand how LLM research has evolved as scientists discover and study these systems' weaknesses and constraints.

This is a systematic literature review, a type of research paper that surveys and analyzes existing studies on differential privacy (a mathematical technique that adds carefully measured noise to data to protect individual privacy) in machine learning. The review examines how researchers are applying differential privacy to train AI models while keeping personal information safe from being extracted or misused.

This academic survey paper categorizes and describes different privacy concerns and system designs in collaborative deep learning (machine learning where multiple parties train models together while keeping their data private). The paper creates a taxonomy, which is a systematic classification scheme, to help organize the various approaches and challenges in this field.

Dagster had a SQL injection vulnerability (a security flaw where attackers can insert malicious SQL commands into database queries) in its database I/O managers (tools that read and write data to databases like DuckDB, Snowflake, and BigQuery). Users with permission to add dynamic partitions (flexible data groupings) could create partition keys that contained SQL commands, which would then execute against the database with the I/O manager's credentials, potentially allowing unauthorized data access or modification.

OpenTelemetry.Exporter.Jaeger has a memory exhaustion vulnerability where internal pooled lists (reusable memory structures) can grow too large based on big payloads and stay oversized for future use, potentially causing denial of service (making a system unavailable). However, the developers have no plans to fix this because the Jaeger exporter was deprecated in 2023.

Flowise versions up to 3.0.13 have a remote code execution vulnerability in the Airtable Agent node where user input is sent to an LLM (large language model, an AI that generates text) to generate Python code, which is then executed without proper sandboxing. An attacker can craft malicious prompts that trick the LLM into generating code containing dangerous commands (like imports or system operations) that bypass the validation checks, allowing them to run arbitrary code on the server without needing to log in.