A new paper titled “SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning” introduces a lightweight way to train Vision-Language-Action (VLA) robots to handle complex, multi-step tasks — even with very little data.
Using just one demonstration per task, the system boosts long-horizon task success from 17.3% to 91.7% on the LIBERO-Long benchmark — a massive leap in efficiency.
How It Works
Traditional supervised training for robots needs thousands of human demonstrations, which are expensive and often fail in new layouts.
SimpleVLA-RL instead learns online using only a binary reward signal:
- 1 for success, 0 for failure — no complicated reward design.
The model samples multiple action sequences, runs them, and uses Group Relative Policy Optimization (GRPO) to favor the better ones. It also explores harder tasks by adjusting sampling temperature and clipping dynamically.
Results and Real-World Impact
Across LIBERO and RoboTwin, SimpleVLA-RL dramatically improves success on long tasks and generalizes to new layouts, objects, and goals.
Even in real-world robots, policies trained only in simulation improve from 17.5% to 38.5% success — without using any real robot data.
Interestingly, the model learns creative strategies like “pushcut,” realizing it can push an object instead of grasping it when that still achieves the goal.
However, the system still needs a base policy with minimal success — otherwise, the binary reward offers no learning signal.
