{"data":{"id":"a96c0084-b291-45f2-b9f8-0ab2defa6f2f","title":"How enabling two settings tripled our scores on the ARC-AGI-3 benchmark","summary":"A team found that GPT-5.6 Sol's performance on ARC-AGI-3, a benchmark testing AI agents' ability to learn and reason about unfamiliar 2D puzzle games, improved dramatically from 13.3% to 38.3% by enabling two API settings: retained reasoning (keeping the AI's internal thoughts between actions) and compaction (a token optimization technique, where tokens are the basic units of text the AI processes). The benchmark's original harness discarded the model's private reasoning after each action and used a rolling truncation window (removing older history as new information arrived), preventing the AI from remembering its past thinking and learning effectively.","solution":"The source explicitly mentions the fix: implement the ARC-AGI-3 harness with the Responses API, which 'makes it easy to manage context: for GPT-5.6, passing the previous response ID automatically retains reasoning across tool calls and turns.' The text states the team enabled 'retained reasoning and compaction' settings 'used in ChatGPT and Codex' to achieve the performance improvement.","labels":["research"],"sourceUrl":"https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores","publishedAt":"2026-07-29T15:00:00.000Z","cveId":null,"cweIds":null,"cvssScore":null,"cvssSeverity":null,"severity":"info","attackType":[],"issueType":"news","affectedPackages":null,"affectedVendors":["OpenAI"],"affectedVendorsRaw":["OpenAI","GPT-5.6 Sol","ChatGPT","Codex"],"classifierModel":"claude-haiku-4-5-20251001","classifierPromptVersion":"v3","cvssVector":null,"attackVector":null,"attackComplexity":null,"privilegesRequired":null,"userInteraction":null,"exploitMaturity":null,"epssScore":null,"patchAvailable":null,"disclosureDate":"2026-07-29T15:00:00.000Z","capecIds":null,"crossRefCount":0,"attackSophistication":"moderate","impactType":null,"aiComponentTargeted":"inference","llmSpecific":true,"classifierConfidence":0.85,"researchCategory":null,"atlasIds":null}}