Skip to content
InfoNewsLLM-specific

Perplexity and Intelligence

Published
Record updated
View JSON

Summary

OpenAI argues that perplexity, the average next-token prediction error on a held-out dataset, can serve as a shared metric for comparing pretraining progress across AI developers and informing industry safety standards. The post publishes OpenAI's internal perplexity measurements, which it presents as a first step toward shared pacing efforts. It links the log-perplexity thresholds of successive GPT models to emergent capabilities, including fluent conversation, chain-of-thought reasoning, and transformative capabilities in cybersecurity and mathematics below 0.8.