Segmenting Robot Video into Actionable Subtasks

Researchers introduced WGO-Bench, a benchmark for automatically segmenting robot videos into actionable subtasks, which is critical for training robots on complex long-horizon tasks. Through 60+ experiments, they found that Gemini models significantly outperform other AI systems for this task, with their best end-to-end pipeline achieving 0.168 F1 and costing $2.64 per video hour—roughly 19 times cheaper than human annotation. The open-source pipeline is implemented in Refiner and addresses a key bottleneck in robotics research as video data collection scales.