Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models
Patrik Wolf, Thomas Kleine Buening, Andreas Krause, Celestine Mendler-Dünner
Read on arXiv →
Key claim
LLMs struggle with statistical self-consistency in estimates.
In plain English
The problem at hand is that in-context learning in LLMs is often seen as conditional inference, but models frequently violate basic probabilistic principles. This paper identifies that LLMs struggle to aggregate estimates from subpopulations into accurate population-level predictions. By demonstrating that more detailed prompts can lead to better estimates, it suggests a new way to evaluate LLM performance based on statistical self-consistency. Builders might care because understanding these limitations can inform better model design and application.
Introduces a new framework for evaluating LLM consistency using probabilistic principles.
Employs rigorous evaluation across multiple models and tasks, though some results may vary.
Deep reliability assessment
The methodology supports the claim that LLMs often violate basic consistency properties, but it may overclaim the general applicability of statistical self-consistency as a universal evaluation criterion.
Reproducibility
No open source code or dataset is mentioned in the paper.
Key figure
Figure 1 illustrates the process of reconstructing aggregate estimates from subpopulation estimates using a binary conditioning tree and the law of total probability.