AI Sec Watch

OpenAI updated ChatGPT to better recognize warning signs of harm by analyzing context within and across conversations, particularly for suicide, self-harm, and harm-to-others scenarios. The system now uses safety summaries (short notes about earlier safety-relevant context) and improved training to distinguish between safe interactions and rare high-risk situations, allowing ChatGPT to respond more carefully through de-escalation, refusal, or redirection to support resources. These improvements were developed in collaboration with mental health experts over more than two years.

Two brothers fired from a hosting company that served 45+ US government agencies used an AI chatbot to help them delete customer databases and cover their tracks, asking it questions like how to clear system logs from SQL servers. The incident highlights that organizations need stronger controls to prevent insider attacks (damage from current or former employees) and must implement better safeguards to prevent AI tools from being misused for destructive purposes.

Microsoft CEO Satya Nadella worried that OpenAI could become more dominant than Microsoft itself, similar to how Microsoft once overtook IBM in the 1980s. Court testimony revealed that Microsoft invested over $100 billion in OpenAI through investments, infrastructure, and hosting costs, and by the end of 2025, about 45% of Microsoft's cloud business obligations were tied to OpenAI, showing how dependent the company had become on its AI partner.

SQLBot is a Text-to-SQL system (software that converts natural language questions into database queries) that uses large language models and RAG (retrieval-augmented generation, where the AI pulls in external documents to answer questions). Before version 1.8.0, it had an IDOR vulnerability (insecure direct object reference, where an attacker can access resources belonging to other users by manipulating request parameters), allowing attackers to access and modify database schemas and data from other workspaces or organizations.

Microsoft Edge is updating its Copilot AI chatbot to access information from all your open browser tabs, letting you ask questions about tab content, compare products, and summarize articles. Users can choose which features to enable or disable, and Microsoft is replacing the older Copilot Mode (which had agentic features like booking reservations) with this new tab-aware version.

Palo Alto Networks warns that hackers are increasingly using AI models to find and exploit software vulnerabilities (weaknesses in code that attackers can use), and companies have only 3-5 months to strengthen their defenses before AI-driven attacks become common. Security teams are under pressure as more sophisticated AI models make it easier for attackers to discover previously unknown vulnerabilities faster than companies can fix them.

AI chatbots like Google's Gemini and ChatGPT are accidentally revealing people's real phone numbers in their responses, sometimes giving out correct personal information and sometimes generating plausible-sounding but wrong numbers that still reach innocent people. Experts believe this happens because of personally identifiable information (PII, real details about people) in the training data (the information used to teach the AI), though the exact mechanism is unclear. The problem appears widespread and difficult to stop, with privacy removal companies reporting a 400% increase in requests about AI-related privacy concerns over the last seven months.

GitHub Copilot CLI (an AI tool that helps developers write code from the command line) has a security vulnerability in versions before 1.0.43 where a malicious bare git repository (a special type of git storage folder with no working files) hidden in a project can trick the tool into running harmful commands. An attacker can exploit git's automatic discovery of these repositories and use configuration keys like core.fsmonitor (settings that tell git what commands to run during normal operations) to execute arbitrary code without the user knowing.

In Vercel CLI versions 50.16.0 to 52.0.0, when running in non-interactive mode (a mode where the tool runs without user interaction, often used in CI/CD systems or with AI agents), authentication tokens (secret credentials that prove your identity) could be accidentally included in plain text within JSON suggestions that the tool outputs. This means the token could be exposed in logs or agent records where it shouldn't be visible.