InfoResearchIndustryLLM-specific
Beyond g-mean²: complementary metrics for monitorability
- Published
- Record updated
Summary
The post introduces complementary metrics for evaluating monitorability beyond the dataset-level g-mean² measure, which scores how well a monitor discriminates target behavior across outcome, process and intervention evaluations. It argues that g-mean² alone leaves open whether model comparisons hold on shared task subsets and how a monitor performs when used as a deployment control that rejects outputs and allows retries. It uses post-control risk to address this, citing an example in which a model trained against a CoT monitor produced more accepted hacks under monitor-gated rejection sampling.
Related items
- InfoJudge dismisses Sam Altman’s sister’s sexual abuse lawsuitSame vendor · The Verge (AI)
- InfoOpenAI CEO Sam Altman wins dismissal of sister's lawsuitSame vendor · BBC Technology
- InfoEx-OpenAI Engineer Detained After Planting Secret Recorder at HQSame vendor · Dark Reading
- InfoAmericans, this is your chance to slap down the tech broligarchsSame vendor · The Guardian Technology
- InfoOpenAI says GPT-6.1 Sol is dramatically better at migrating legacy codeSame vendor · BleepingComputer