Multimodal Model Diffing for Feature Discovery and Control

Imagine you're developing a multimodal AI that can understand both text and images, like a virtual assistant that can interpret photos and respond to questions about them. The challenge lies in understanding how these models make decisions based on their training, especially when they behave unexpectedly. Current methods often rely on post-hoc analysis, which can miss the nuances of how different features interact and change during training, leading to issues like misinterpretation of visual data or unsafe outputs. This is what's called feature isolation failure, where you can't pinpoint which aspects of the model's understanding are responsible for its behavior.

To address this, MMDiff offers a framework that allows for deeper inspection and control of multimodal models by using sparse autoencoders (SAEs). It enables builders to isolate specific features that have been altered through multimodal training, detect task-specific features, and even control these features to improve model performance. By training SAEs across various multimodal model families, MMDiff reveals that certain features can be causally linked to performance on tasks like visual-spatial understanding and OCR. This means that instead of just interpreting model behavior, developers can actively steer it, enhancing safety and capability in real-world applications.

MMDiff introduces a novel framework for feature-level control in multimodal models.

The evaluation includes multiple MLLM families and diverse tasks, providing solid evidence of effectiveness.