The classifier judged this item not relevant to AI security, so it is left out of lists, feeds, the API and dataset releases. If that is wrong, report a correction on this page.
A Simple Unified Uncertainty-Guided Framework for Offline-to-Online Reinforcement Learning
- Published
- Record updated
Summary
Offline reinforcement learning agents trained on fixed datasets often perform suboptimally, so they need further finetuning through online interaction. This paper proposes SUNG, a simple unified uncertainty-guided framework for offline-to-online RL. SUNG estimates uncertainty with a VAE-based state-action visitation density estimator, uses optimistic exploration, and adaptively applies conservative offline objectives to high-uncertainty samples and standard online objectives to low-uncertainty ones. It reports state-of-the-art online finetuning performance across D4RL environments and datasets.