CoVal: Learning values-aware rubrics from the crowd

Summary

OpenAI researchers released CoVal, an experimental dataset of crowd-written, prompt-specific rubrics that show why people prefer one model response over another, in addition to which response they chose. CoVal-full keeps the raw, sometimes conflicting criteria, while CoVal-core keeps 4 highly rated, mutually compatible criteria per prompt. The authors state that the rubrics reflect surveyed participants' views, not OpenAI's, and do not represent what all people want from AI.

Related items

• InfoRogue Anthropic AI agent gave police fake tip in unsolved murder caseSame vendor · BBC Technology
• InfoOpenAI Fires 3 Safety Researchers in Dispute Over AI RisksSame vendor · SecurityWeek
• Info‘Pure insanity’: Mathematicians will need years to make sense of OpenAI’s latest dropSame vendor · The Verge (AI)
• InfoOpenAI reports three new incidents of misalignmentSame vendor · CSO Online
• InfoA new feature for my blog, built using my voiceSame vendor · Simon Willison's Weblog