AI Sec Watch

Singular Bank built Singularity, an internal AI assistant powered by ChatGPT and Codex (OpenAI's code-generation model), to help bankers quickly analyze client investment portfolios and prepare communications. The system saves bankers 60-90 minutes daily by automating tasks like portfolio analysis, meeting preparation, and follow-up drafting, allowing them to spend more time advising clients and building relationships.

vLLM (a system for running large language models) has a vulnerability where specially crafted text prompts containing multimodal placeholder tokens (sequences that represent images or videos) without actual image or video data cause the system to crash with an IndexError (a programming error when accessing data that doesn't exist). An unauthenticated attacker can send a single malicious request to a vLLM server to trigger a denial of service attack (making the service unavailable), affecting any deployment that runs vision-capable language models.

The `discover_pipeline_files()` function in ciguard (a tool used by AI agents to scan code repositories) followed symlinks (shortcuts that point to other directories) without proper restrictions, allowing an attacker to trick it into reading sensitive files outside the intended scan directory. An AI agent scanning a malicious folder with planted symlinks could accidentally expose secrets from system directories like ~/.aws/ or /etc/.

The OpAMP client (a component for managing telemetry agents) reads HTTP responses without limiting how much data it accepts, which could allow an attacker controlling the server to send extremely large responses and exhaust the application's memory, causing it to crash. This vulnerability only affects applications where the OpAMP server is untrusted or could be intercepted by a network attacker.

Google has updated Gemini for Home to version 3.1, which improves the AI assistant's ability to handle complex, multi-step tasks and combine multiple requests in a single command. The update also enhances Gemini's understanding of natural language (how humans normally speak), device identification, and management of calendar events. These improvements follow reports of bugs in the smart home assistant.

Attackers are using supply-chain attacks (compromising software components that developers rely on) to target AI coding agents, which automatically scan package registries like NPM and PyPI for dependencies to include in projects. A North Korean group called Famous Chollima launched the PromptMink campaign, using fake packages with legitimate-sounding names and descriptions, along with hidden malicious code, to trick AI agents into installing malware that steals information and grants attackers remote access to developers' computers.

SQLBot is a Text-to-SQL system (software that converts natural language questions into SQL database queries) that uses large language models and RAG (retrieval-augmented generation, where the AI pulls in external data to help answer questions). In versions 1.7.0 and earlier, it has a prompt injection vulnerability (where an attacker hides malicious instructions in their input to trick the AI), because user questions are directly inserted into the AI prompt without filtering, and the resulting SQL commands are executed without checking if they're safe. An attacker with access can craft a malicious question to make the system run harmful SQL commands, potentially allowing remote code execution (the ability to run commands on a system they don't own) when using PostgreSQL.

Apple is planning to let users choose their preferred AI model for Apple Intelligence features in upcoming operating systems (iOS 27, iPadOS 27, and macOS 27) expected this fall. Third-party AI models, called "Extensions," will be able to power features like Siri, Writing Tools, and Image Playground across the system. Users will also be able to assign different Siri voices to different AI models.

CISA (US Cybersecurity and Infrastructure Security Agency) is considering reducing the time government agencies have to fix critical vulnerabilities from 14 days to 3 days, partly due to concerns that AI models like Claude will help attackers find and exploit serious security flaws more quickly. Currently, the most urgent vulnerabilities (zero-days, which are flaws being actively exploited with no patch available) require fixes within 24-72 hours, while other critical vulnerabilities under active exploitation have 14 days. Security experts have mixed views on whether a 3-day timeline is realistic, with many concerned it doesn't allow enough time for proper testing before deploying patches.