From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model

Imagine you're building a robot that needs to perform tasks in various environments, but the camera setup you trained it with often changes in the real world. This can lead to problems because existing systems rely on knowing exactly where the camera is positioned, which can be tricky and fragile. When the camera moves or is remounted, the robot struggles to adapt, leading to failures in task execution. This is what's called view robustness failure, where the robot can't handle unexpected camera perspectives effectively.

The solution proposed in this paper is to shift the focus from telling the robot where the camera is to allowing it to figure that out on its own. The new model, called Camera-Centric VLA (CamVLA), does this by predicting actions based on the camera's perspective without needing to know its exact position. It generates actions in a way that is independent of the camera's geometry, which means it can work with just a single image and task instruction at deployment. This decoupling allows the robot to operate more flexibly and reliably in real-world scenarios.

What sets CamVLA apart from previous methods is its ability to improve success rates across various unseen viewpoints without requiring complex calibration or depth information. This means that for builders deploying robots in dynamic environments, this approach simplifies the process and enhances the robot's adaptability, making it a practical choice for real-world applications.

The approach introduces a new way to handle camera variations in robot deployment, significantly extending existing methods.

The evaluations in both simulation and real-world scenarios provide solid evidence for the claims made.