InfoResearchPeer-reviewedLLM-specific
Evaluating Large Language Models on Named Entity Recognition
- Published
- Record updated
Summary
Researchers evaluate twenty-eight representative LLMs on thirteen named entity recognition (NER) datasets across five domains, with parameter counts from 3 billion to 175 billion. They assess supervised fine-tuning, parameter scale, hallucinations and prompt designs using an LLM-based NER framework (LLM-NER) with a Recognition phase and a Check phase. The study finds that fine-tuning improves instruction following, performance rises with parameter scale, hallucinations appear in all evaluated models, and all models are sensitive to prompt design.
Mitigation
Guiding LLMs to check their outputs, via the Check phase of LLM-NER, is described as a feasible way to alleviate hallucinations.