When Annotators Disagree: A Principled Approach to Learning With Noisy Labels

When multiple humans label the same data for training AI models, they often disagree, creating noise (errors) in the labels that can hurt model performance. This paper shows how to measure how much annotators agree with each other, use that agreement to estimate what kind of label noise exists, and then train models that work better despite the noisy labels. The researchers also provide theoretical guarantees about how well these models will perform on new data.