Making it easier for websites to express AI training preferences

Summary

OpenAI will honor a shared signal in robots.txt that lets website operators state their preferences about using their content for foundational generative AI training. The signal lets operators set one preference for multiple participating providers instead of maintaining separate rules per crawler. OpenAI aims to implement it in the coming months and says existing robots.txt crawler controls will still apply.

Related items

• InfoSam Altman says it’s better for RAM to be in a data center than on your deskSame vendor · The Verge (AI)
• InfoThe biggest names in AI are backing a new way to opt out of scrapingSame vendor · The Verge (AI)
• InfoGhost in the machine: how AI is haunting the search for the supernaturalSame vendor · The Guardian Technology
• InfoSam Altman sympathizes with mathematicians, but says OpenAI’s math onslaught won’t stopSame vendor · The Verge (AI)
• InfoWikipedia’s operator says a bad AI copyright ruling could threaten the open webSame vendor · The Verge (AI)