LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training
Shun Lei, Huaicheng Zhang, Dapeng Wu, Yaoxun Xu, Lishi Zuo, Wei Tan, Hangting Chen, Guangzheng Li, Jianwei Yu, Zhiyong Wu, Dong Yu
Read on arXiv →
Key claim
LeVo 2 outperforms existing systems in song generation quality.
In plain English
Imagine you're trying to create a full-length song that sounds coherent and musical, while also ensuring that the vocals and instruments work well together. This is a tough challenge because existing systems often struggle with either keeping the overall sound cohesive or providing detailed sound quality for each track. When they focus on one aspect, they tend to lose out on the other, leading to songs that might sound good in parts but not as a whole. This is what's called a structural trade-off in song generation.
To tackle this, LeVo 2 introduces a clever way to manage these competing needs. It starts by predicting the overall structure of the song using a mixed-token approach, which helps in planning the song's semantics. Then, it refines the details for vocals and instruments in parallel, ensuring that both elements are well-coordinated. Additionally, it uses a diffusion-based Music Codec to reconstruct the final audio, which helps in maintaining high sound quality.
What sets LeVo 2 apart from previous methods is its aesthetics-guided training schedule. This means that during training, it uses an automated system to evaluate the musicality of the songs, which helps in aligning the model's outputs with what sounds good. The results show that LeVo 2 not only improves the quality of song generation but also enhances controllability and musicality, outperforming many existing systems. For anyone building music generation tools, this approach offers a more balanced and effective way to create songs that sound great from start to finish.
LeVo 2 introduces a hybrid approach that effectively balances semantic planning and track-specific refinement, advancing the state of the art in song generation.
The paper provides strong experimental validation through expert listening tests and objective evaluations against open-source baselines.
Deep reliability assessment
The methodology supports the claim that the hierarchical mixed-token-to-dual-track architecture and progressive DPO schedule improve their own LeVo 2 variants under MOS, Gemini-based prompt metrics, ASR, and PER evaluations. The stronger claim that it approaches commercial systems is harder to assess from the provided excerpt because the commercial comparison details, test set construction, rater protocol, and potential training-data overlap are not visible.
Reproducibility
No open-source code, model weights, or dataset release is mentioned in the provided abstract, introduction, results, limitations, or conclusion excerpts. The paper describes large-scale proprietary-style training data, automated aesthetic labeling, expert listening tests, and Gemini-based evaluation, but not enough detail to fully reproduce the system.
Key figure
Figure 1 shows LeVo 2 as a hybrid LLM-diffusion pipeline where LeLM predicts mixed semantic tokens and dual-track vocal/accompaniment tokens, a diffusion Music Codec reconstructs audio, and training proceeds through aesthetic-conditioned pretraining, progressive post-training, and modular track-specific extension.