Skip to content
InfoResearchPeer-reviewedLLM-specific

Ensemble Learning for Large Language Models in Text and Code Generation: A Survey

Published
Record updated
View JSON

Summary

This survey reviews ensemble techniques for large language models in text and code generation, motivated by inconsistent outputs, biases and data privacy limits of closed-source models. It categorizes LLM ensembles into seven methods: weight merging, knowledge fusion, mixture-of-experts, reward ensemble, output ensemble, routing, and cascading. The authors report benefits including improved diversity representation, enhanced output quality, and greater application flexibility.