InfoResearchPeer-reviewedLLM-specific
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey
- Published
- Record updated
Summary
This survey reviews ensemble techniques for large language models in text and code generation, motivated by inconsistent outputs, biases and data privacy limits of closed-source models. It categorizes LLM ensembles into seven methods: weight merging, knowledge fusion, mixture-of-experts, reward ensemble, output ensemble, routing, and cascading. The authors report benefits including improved diversity representation, enhanced output quality, and greater application flexibility.