The Bitter Lesson Comes to Robotics

Summary

Researchers evaluated seven frontier multimodal language models, at multiple reasoning-effort settings, on six manipulation tasks from the RoboDojo benchmark, testing whether they can directly command physical robotic arms. Current models can perform a variety of manipulation tasks but are not yet precise or efficient enough to finish them within a practical action budget. Between July and September 2026, performance on the benchmark doubled every 44.5 days, the best execution rate rose 4.4 times, and the lowest execution cost fell 6.2 times.

Topics

• Robotics and embodied AI