Consilience for Verifier-Free Test-Time Scaling

Imagine you're building a large language model that needs to solve complex problems, like advanced math or generating code. The challenge is ensuring that the model not only provides answers but does so reliably, especially when it encounters difficult tasks. Currently, many methods rely on confidence scores to evaluate the quality of the model's outputs. However, this approach often leads to issues where high confidence can mislead the model into favoring incorrect answers, a problem known as overconfidence. This is particularly problematic in complex scenarios where exploration is crucial for finding the right solution.

To tackle this, the authors propose a new framework called consilience, which emphasizes the importance of how confidence evolves over time. Instead of simply relying on a final confidence score, consilience encourages a pattern where the model starts with lower confidence, explores various possibilities, and then converges to a high-confidence answer. This method actively penalizes high initial confidence, promoting a more exploratory approach to problem-solving. The results show that consilience outperforms existing methods in challenging tasks, suggesting that a focus on the trajectory of confidence can lead to more robust reasoning in language models.

The introduction of consilience offers a new perspective on confidence trajectories in reasoning.

The experiments validate the approach across diverse complex tasks, though more baselines could strengthen claims.