JEPA-Anything: Learning Predictive Models across Different Worlds

Imagine you're developing an AI that needs to understand and predict complex interactions in various fields, from biology to weather systems. The challenge lies in creating models that can generalize across these different domains without being overly specialized. Traditional predictive models often fail when applied outside their training context, leading to poor performance in real-world scenarios — this is what's called domain specificity. They struggle to adapt when faced with new types of data or unexpected situations, which can result in inaccurate predictions and missed opportunities for intervention. To address this, JEPA-Anything proposes a framework that leverages orthogonal predictive factorization (OPF). This approach allows the model to break down complex predictions into simpler, complementary factors that can be learned independently and then recombined. By doing so, it enhances the model's ability to generalize across various domains, as demonstrated through extensive evaluations in areas like vision, clinical trajectories, and molecular dynamics. Compared to previous models, JEPA-Anything not only improves prediction accuracy but also supports experimental interventions, bridging the gap between theoretical modeling and practical applications. For builders, this means a more robust tool for creating AI systems that can operate effectively in diverse and dynamic environments.

JEPA-Anything introduces a novel framework for domain-agnostic world modeling.

The evaluation across multiple domains and tasks shows solid performance improvements over baselines.