Pictura: Perspective-View Self-Play at Scale for Driving

Imagine you're developing an autonomous vehicle that needs to navigate complex environments using only its onboard cameras. Traditional methods often rely on privileged information, like exact positions and velocities of other vehicles, which isn't available in real-world scenarios. This reliance creates a gap in understanding, as the vehicle's decisions are based on incomplete information, leading to potential failures in dynamic situations — this is what's called the representation gap. When agents are trained using privileged data, they may struggle to generalize their learned behaviors to real-world conditions where such data isn't accessible.

To address this, the authors propose a new approach called perspective-view self-play, which trains agents using only the egocentric views they would actually encounter in the field. They developed Pictura, a high-performance multi-agent driving simulator that generates these views in real-time, allowing agents to learn directly from their own perspective. By training a policy named Alberti through self-play with Proximal Policy Optimization (PPO) over 50 billion agent steps, they achieved performance that closely matches agents trained with privileged data, even outperforming them in certain scenarios. This shift means that builders can now create more robust driving policies that are better suited for real-world applications, without the need for idealized training conditions.

The introduction of perspective-view self-play in driving simulations represents a significant advancement in training methodologies.

The results are backed by extensive training and performance comparisons against privileged observations.