DanceOPD: On-Policy Generative Field Distillation

Imagine you're trying to build a single model that can do everything related to image generation — from creating images from text to editing those images in various ways. The challenge is that these tasks often conflict with each other. For example, if you focus on editing an image, it might hurt the model's ability to generate images from text. This is a common issue in the field, where different capabilities can interfere with one another, leading to subpar results. This problem is known as capability conflict.

To address this, the authors propose a new method called DanceOPD. Instead of trying to make one model handle everything at once, DanceOPD routes each image generation task to a specific capability field. It uses a technique that allows the model to learn from its own experiences, focusing on one task at a time while still being aware of the others. This way, the model can improve its performance on each task without degrading the overall quality of the images it generates.

What sets this work apart from previous methods is its ability to effectively compose multiple capabilities while preserving the quality of the generated images. The experiments show that DanceOPD not only enhances the performance of text-to-image generation but also improves editing capabilities. For anyone building image generation systems, this approach offers a practical solution to a common problem, making it easier to create models that can handle a variety of tasks without compromising on quality.

The approach introduces a new framework for effectively composing multiple image generation capabilities.

The experiments demonstrate improvements across various tasks, though more extensive baselines could strengthen claims.