A Survey of Progress in LLM Alignment From the Perspective of Reward Design

This survey examines how rewards (scoring systems that guide AI behavior) are designed to align LLMs (large language models, or AI systems trained on massive amounts of text) with what humans want them to do. The review organizes the field by asking how rewards are mathematically defined, how they are built using different data sources and methods, how they work with different training approaches like reinforcement learning from human feedback (RLHF, a technique where humans rate AI outputs to improve performance), and how they are tested for safety and effectiveness.