HPRO: Hierarchical Progressive Reward Optimization via Preference Extraction for Emotional Text-to-Speech

Imagine you're building a text-to-speech system that sounds natural and conveys emotions effectively. The challenge is that traditional methods often lead to a bland, averaged-out voice that lacks emotional depth. This happens because the way these systems are trained can create conflicts between the content of the speech and the emotional tone, which is what we call information conflict. Additionally, when trying to optimize for emotions, the feedback often comes in sparse, high-level rewards that don't translate well to the detailed, frame-by-frame adjustments needed in speech generation — this is known as the scale gap.

To tackle these issues, the authors propose a new approach called HPRO, which stands for hierarchical progressive reward optimization. The idea is to separate the emotional aspects from the content in a structured way, allowing the system to focus on optimizing each part without them interfering with each other. They introduce a new reward model, the HD-Emo codec, which helps in isolating emotional preferences from the actual content of the speech. By progressively aligning objectives at different levels — from individual frames to entire sentences — HPRO effectively bridges the gap between high-level emotional goals and the detailed requirements of speech generation.

The results show that HPRO not only enhances the emotional expressiveness of the generated speech but also keeps it clear and understandable. For anyone building text-to-speech systems, this means you can create voices that sound more human and convey feelings better, which is crucial for applications like virtual assistants, audiobooks, or any interactive voice systems.

The proposed HPRO framework introduces a novel approach to optimizing emotional expressiveness in TTS models.

The experiments show significant improvements in emotional expressiveness while maintaining intelligibility, supported by a clear methodology.