A shared playbook for trustworthy third party evaluations

This document outlines best practices for evaluating frontier AI models (advanced AI systems at the cutting edge of capability) through independent third-party assessments. Modern frontier models are more complex than simple chatbots because they can use tools, maintain information across multiple steps, and operate within larger workflows, so evaluations must account for the "harness" (the surrounding setup and environment) that can significantly affect performance. Evaluation reports should clearly state what claim is being tested (such as whether a model can perform a capability, how robust its safety features are, or how it compares to other models) and provide evidence that the results are valid by addressing potential issues like reward hacking (exploiting shortcuts in scoring), contamination (overperforming due to exposure to similar tasks in training data), and sandbagging (deliberately underperforming when aware of being evaluated).