Breaking the Uniformity Trap: Scaling Video Diffusion Model via SplitMoE
Yu Xu, Yuxin Zhang, Xiao Yang, Haotian Yang, Yizhi Wang, Xinwei Huang, Minxuan Lin, Angtian Wang, Chongyang Ma, Fan Tang
Read on arXiv →
Key claim
SplitMoE improves video generation through semantic expert routing.
In plain English
Imagine you're developing a system that generates videos, like creating animations or synthesizing scenes from text. The challenge lies in effectively managing the vast amount of visual information, especially when it comes to understanding the context and maintaining coherence across frames. Current methods often treat each piece of data independently, leading to a situation where related visual elements get scattered across different processing units, which is inefficient and can result in disjointed outputs. This issue is known as routing fragmentation, where the system fails to leverage the inherent relationships in the data, leading to a lack of structural integrity in the generated content.
To tackle this, the authors propose a new architecture called SplitMoE, which organizes the processing units into two distinct roles: semantic experts that focus on high-level concepts and generic experts that handle the finer details of the visuals. This bifurcation allows the system to route information based on its semantic meaning rather than arbitrary balancing rules, which improves coherence and quality in video generation. By employing techniques like prototype-guided routing, SplitMoE not only enhances the efficiency of expert usage but also accelerates convergence during training. Compared to traditional methods, this approach offers a more intuitive way to scale video generation models, making it a valuable reference for future developments in the field.
The approach introduces a novel architecture that addresses specific challenges in video data processing.
The results are backed by extensive evaluations against standard benchmarks, demonstrating clear improvements.
Deep reliability assessment
The methodology supports improved routing coherence and video generation quality by addressing the uniformity trap in MoE architectures, but the claims of scalability and quality improvements may be overclaimed without extensive real-world testing.
Reproducibility
yes, the project page is provided: https://yuci-gpt.github.io/SplitMoE/
Key figure
Figure 1 illustrates how SplitMoE scales video diffusion models by introducing semantic-aligned expert partitioning, reducing spatiotemporal fragmentation, and generating higher-quality videos compared to dense fine-tuning and WAN with a standard MoE.