Democratic ICAI: Debating Our Way to Steering Principles from Preferences

Imagine you're trying to build a system that understands human preferences, like what makes a good movie or a great piece of art. Traditionally, people would just ask for a simple choice between options, but that doesn't capture the complex reasons behind those choices. This is where things can go wrong: you might miss out on important factors that influence decisions, leading to a system that doesn't really understand what people want. This is what's called a lack of interpretability in preference-based systems.

To address this, Democratic ICAI takes a fresh approach. Instead of relying on a single choice, it gathers multiple competing viewpoints through structured debates among different personas. This method captures a wider range of considerations and nuances that shape preferences. By summarizing these diverse rationales into clear principles, it helps guide decision-making in a more informed way.

What sets Democratic ICAI apart from previous methods is its ability to produce richer signals about preferences, leading to better predictions in creative tasks. In practical terms, if you're building a system that needs to understand and predict human choices, this approach could significantly improve how well it aligns with actual human preferences, making it more effective in real-world applications.

Democratic ICAI introduces a new method for preference alignment through structured persona debate.

The experiments show improved preference prediction across multiple benchmarks, supporting the claims made.