AI Sec Watch

RAGFlow, an open-source RAG (retrieval-augmented generation, where an AI pulls in external documents to answer questions) engine, has a Jinja2 template injection vulnerability (a flaw where untrusted data gets processed as code in a templating system) in version 0.24.0 and earlier. Any registered user can exploit this flaw in the prompt generator to run arbitrary OS commands (any commands they want) on the server by creating a Canvas workflow with specific components.

The European Union wants to increase discussions with the U.S. about advanced AI models that have cyber capabilities, particularly after Anthropic's Mythos model (a very powerful AI system) raised concerns about AI-powered cyberattacks. The EU has not yet received access to preview Mythos, and the White House has opposed expanding access to the model beyond the U.S. due to security concerns, though Anthropic says it is developing safeguards and expects to release Mythos-class models to customers within weeks.

Boston Children's Hospital integrated AI (artificial intelligence) across its entire organization as a core part of clinical and operational work, rather than treating it as a separate experiment. By building an enterprise AI layer (a shared, secure internal AI system used across teams) and redesigning workflows in areas like supply chain and surgical scheduling, the hospital has diagnosed over 40 previously unresolved rare conditions, saved approximately 60,000 hours of staff time, and enabled more than one-third of employees to use AI daily in their work.

Braintrust, an AI observability company, uses Codex (OpenAI's code-generation AI model) to quickly turn customer feature requests into working preview branches in minutes, with half the team adopting it within one month. The speed of Codex enables faster feedback loops with customers and allows engineers to test ideas in real time rather than letting requests sit in a backlog. Codex's ability to handle large amounts of text output without slowing down makes it more effective than other models for this workflow.

Employees are using AI-driven development platforms (vibe coding, where non-programmers build working applications by describing what they want) to quickly build custom applications and connect them to company systems, then publish them on the public internet without involving security teams or implementing basic access controls. A study found over 2,000 such exposed applications containing sensitive data across major companies, sitting unprotected because traditional security tools like EDR (endpoint detection and response, software that monitors what happens on company devices) and DLP (data loss prevention, software that blocks sensitive information from leaving the company) were designed to catch different types of threats and don't detect these cloud-to-cloud connections or applications built in web browsers.

Adobe's Firefly AI Assistant is a conversational AI agent designed to automate tasks within Adobe's design software while keeping users in control of the creative process, unlike traditional AI image generators that work independently. The assistant acts as a multitasking middleman that can operate design apps on behalf of users, though early testing suggests the results are not particularly impressive despite the tool's thoughtful approach to preserving creative control.

Big tech companies are legally challenging GDPR (General Data Protection Regulation, Europe's data protection law) fines, with nearly 40% of the €7.1 billion in fines announced over eight years either annulled or under appeal. While GDPR successfully established a global 72-hour breach notification standard (the requirement that organizations tell people within three days if their data is stolen), experts note the framework has structural weaknesses that companies exploit in court, and upcoming AI regulations may face similar challenges.

Organizations are rapidly adopting unauthorized AI tools without proper security oversight, creating 'shadow AI' (unsanctioned AI use that bypasses governance controls) that exposes sensitive data and creates new attack surfaces. Traditional security tools like firewalls and Zero Trust architecture (a security model that requires verification for every access request) cannot detect AI-specific threats such as prompt injection (tricking an AI by hiding malicious instructions in its input), leaving companies vulnerable to data leaks, compliance failures, and attacks that exploit AI systems.

OpenAI is launching Rosalind Biodefense, a program that gives vetted developers access to GPT-Rosalind (a reasoning model trained for life sciences) to build defensive tools against biological threats like pandemics. The company is also expanding trusted access to this model for select U.S. government and allied partners working on public health and biodefense, supported by safety measures like capability assessments, expert red teaming, and security controls to prevent misuse.