Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision

Imagine you're building a language model that needs to explain its decisions, like why it chose a certain answer. The challenge is that many models just mimic patterns from their training data without truly understanding their own reasoning. This can lead to explanations that sound good but don't reflect the model's actual thought process — a problem known as superficial imitation. When models are trained to explain their predictions, they often rely on past behaviors or similar models, which can result in explanations that don't align with their current actions. This is what's called a lack of faithful introspection.

What this paper explores is a way to train language models using fixed counterfactual explanations, which are essentially hypothetical scenarios that show how different inputs would lead to different outputs. The authors found that when models are trained this way, they can produce explanations that are surprisingly faithful to their current behaviors, even if those behaviors have changed since the training. This means that the models can track their own shifts in behavior without needing constant updates to their training data.

In practical terms, this approach allows for scalable and generalizable training signals for introspection, which can be particularly useful in applications where understanding model decisions is crucial. By using fixed datasets of counterfactual explanations, builders can enhance the reliability of model outputs without the need for extensive retraining or new labels.

The paper introduces a novel approach to training language models for introspection using fixed counterfactual explanations.

The findings are supported by multiple tasks and demonstrate robustness to label noise, though more extensive baselines could strengthen claims.