Perplexity and Intelligence
OpenAI's Chief Scientist argues that perplexity—a metric measuring how well AI models predict the next word—should be established as an industry standard for tracking AI progress and informing shared safety measures. The company is publishing its internal perplexity measurements from GPT-3.5 onwards to provide transparency, demonstrating that this single metric correlates with major leaps in AI capabilities, from coherent text generation to reasoning and in-context learning. OpenAI believes standardizing perplexity across the industry could create a shared scientific basis for coordinating AI safety efforts and pacing development.