How an assessor could investigate evidence of acceleration from the automation of AI R&D

Summary

This article describes how an assessor could investigate whether AI capabilities progress has accelerated because of AI-driven automation of AI R&D. The assessment would examine capability benchmarks, intermediate indicators such as pretraining loss, and researcher perceptions, and would analyze unreleased models along with the growth of labor, compute, data and algorithmic inputs. It would also assess bottlenecks and whether the methods would reliably detect a dramatic acceleration.