Anuj Bharambe

Reinforcement learning improves vision-language-action models, but it is unclear whether it teaches robots new skills, unlocks skills they could already perform occasionally, or simply makes existing skills more reliable. We compare each RL-trained model with the imitation-trained model it started from, giving both many attempts from identical start states on standard tests of visual, semantic, and execution generalization, with controls that separate the effect of reward from extra training. We then identify which internal changes cause the improvement by transplanting them into the starting model or removing them, across multiple VLAs, benchmarks, and a real robotic arm.

Robot data is scarce, so VLAs are pretrained over the same data many times, yet it is unknown whether this makes them harder to fine-tune, as has been shown for language models. We pretrain VLAs while saving a checkpoint every epoch, fine-tune each one on tasks both close to and far from the pretraining data, including on a real robotic arm, and test when longer pretraining starts to hurt. The goal is practical guidance on when to stop pretraining and how to choose the best starting point for fine-tuning.

Building on our agentic evaluation work, we are studying how well frontier vision-language-action models for robotic arms complete tasks across a wide range of conditions. We report where current models break down and outline the most promising directions for researchers working to improve them.

LLM agents working on long, multi-step tasks can gradually drift from what the user asked for while appearing to work correctly at every step. We are building a lightweight monitoring layer that detects this drift as it happens and corrects the agent before small errors compound.