Vision-Default, Prior-Override: Causal Mechanisms of Perception-Knowledge Conflict in Vision-Language Models

Imagine you're building a system that combines images and text, like a virtual assistant that can understand both what it sees and what it knows. The challenge arises when the visual information conflicts with the knowledge stored in the model. For instance, if the model sees a green apple but knows that apples are typically red, it has to decide which information to trust. This is a common issue in vision-language models, where the way they resolve these conflicts can lead to unreliable outputs. When the model relies too heavily on its memorized knowledge, it might give an answer based on that knowledge instead of what it sees, which can lead to mistakes. This is what's called knowledge bias, and it can be problematic in real-world applications where accuracy is crucial. The authors of this paper took a closer look at how these models work under the hood. They found that visual grounding, or the ability to connect visual inputs with the right knowledge, often happens automatically. However, when the model needs to rely on prior knowledge, it depends on a small number of specific attention heads in the network. By removing these heads, they observed a significant shift in the model's predictions, showing that the model's reliance on knowledge can be quite fragile. This insight is important because it reveals a sparse causal structure that underlies how these models handle conflicts between what they see and what they know. For builders, this means that understanding and potentially modifying these attention heads could lead to more reliable vision-language systems.

The paper introduces a detailed causal analysis of visual grounding in vision-language models, extending prior work significantly.

The findings are supported by comprehensive experiments across multiple model families and ablation studies.