Deepfakes and impersonation

Microsoft plans to add support for third-party deepfake detection solutions and a new impersonation protection feature to Teams meetings. According to Microsoft 365 Roadmap entries, both are in development and are slated for general availability in November after a worldwide rollout.

Frontier AI firm Modulate raised an additional $25 million from Future Ventures, with participation from Hyperplane and Lakestar, bringing its total funding to $60 million. The company builds audio-native AI models for its Velma platform, which combines more than 100 specialized models to detect signals such as emotion, tone, intent and synthetic speech in real time.

Meta's Oversight Board ordered the company to remove two AI-generated deepfake videos from Facebook: one showing a Labour councillor in Scotland making inflammatory comments about refugees, and one falsely depicting a Muslim campaign volunteer giving health advice. The board also said Meta should do more to tackle AI-generated fake imagery.

New York's Manhattan District Attorney's Office seized 12 domains it says were used to share, publish and sell celebrity deepfake videos. Officials say around 1,200 people, overwhelmingly women, were depicted in nonconsensual sexual images and videos on the sites, and investigations into those responsible and the uploaders are ongoing. The seizures were carried out under New York's criminal procedure laws.

Emily Schuman, creator of the lifestyle blog Cupcakes and Cashmere, found an Instagram ad showing her holding GLP-1 drugs for the telehealth company Gala. She had never heard of Gala and had no connection to the ad, which was built from an altered selfie she posted years earlier. Further AI-generated posts appeared showing her promoting Meroda Cosmetics and Superpower.

Apple appears to be developing an iOS feature that verifies when a photo was taken on an iPhone camera. According to 9to5Mac, code references in iOS 27 beta 5 point to an "Apple Reference Image" system that embeds provenance metadata into photos at capture. The feature is not yet live, and a privacy disclosure in the beta says it will be off by default.

The Report Remove service, which blocks intimate images from appearing online, says UK children have reported a rapid increase in digitally manipulated explicit images of themselves. A safety watchdog warned that AI and nudification tools are making such sexualised content easier to produce.

Organized crime groups are running scams at scale and making billions, according to the source. The source names AI-enabled voice cloning, deepfake real-time video overlays, LLM-driven persona management and automated translation as the tools behind the activity.

The EU's AI Act transparency obligations took effect on August 2nd, requiring companies to disclose when people are interacting with AI models and when content has been generated or altered by them. The rules split duties between providers, which develop and market AI systems, and deployers, which use them, and some companies such as Meta and SpaceXAI fall under both.

Google shut down a Google Earth feature launched Thursday that let users edit satellite images with text prompts using AI. Digital Digging's Henk van Ess used it to generate images of refugees near the Mexican border and a bomb crater by a hospital in Gaza. Google had initially said the images carried a digital watermark and that it blocked "image creation on harmful topics."

The European Union launched a new team in Brussels to monitor AI companies for violations of its AI Act, including sexually explicit material, fake photos and videos, and cyber threats to public infrastructure. The AI Act comes into force on Sunday and requires AI companies to label chatbot output and imagery as AI-generated, and Brussels can fine firms or cut off their access to the EU market for breaches.

xAI is suing Terry Wayne Harwood, a South Carolina man, for allegedly using the Grok chatbot to generate child sexual abuse material. The company says he knowingly used Grok to circumvent safeguards, alter nonconsensual images and distribute the material, breaching its policies. Harwood was arrested in February and faces eight felony charges.

Meta is turning off a feature it announced this week after significant backlash. The feature let users generate AI images from content on public Instagram accounts by @-mentioning them, and as originally set up, it allowed content from any public account to be used without the account owner's permission.

Political candidate Jonathan Rinaldi used an AI chatbot on his iPhone to create fake news stories and endorsements he never received. During last October's campaign for a Queens city council seat, he shared a fabricated story styled with a CNN logo, claiming his opponent, incumbent Democrat Lynn Schulman, had dropped out. She had not, and she won in November by a landslide.

The U.S. Department of Justice and Homeland Security Investigations seized the CFAKE.com and SOCFAKE.com websites, which allegedly hosted nonconsensual AI-generated nude images and videos of women, in what appears to be the first publicly announced domain seizure under the TAKE IT DOWN Act. The seizure was part of a coordinated operation with Italy and France, and a suspect was arrested in Nice, France, on June 10. The TAKE IT DOWN Act makes publishing such intimate imagery and digital forgeries without consent a federal crime and requires platforms to remove reported images within 48 hours of a valid victim request.

A UK-based Chinese activist, Apple Peiqing Ni, founder of the China Dissent Network, was targeted on X by deepfake posts portraying her as a sexually promiscuous drug addict. UK police advised her to complain to the platform, but X told her the abuse did not breach its rules.