Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation
Shayan Talaei, Abhinav Chinta, Devvrit Khatri, Amin Karbasi, Azalia Mirhoseini, Amin Saberi
Read on arXiv →
Key claim
D2D reliably detects hidden biases in language models.
In plain English
Imagine you're using a language model to help make important decisions, like recommending products or providing information. The problem is that these models can develop hidden biases, favoring certain brands or viewpoints without any obvious signs. This can happen because the biases are only revealed in specific contexts, making them hard to detect. Current methods struggle to identify these biases, especially when they only show up in certain topics while the model behaves normally in others. This is what's called stealth preferential bias.
To tackle this issue, the authors propose a new method called Distill to Detect (D2D). The idea is to capture the differences between a biased model and its original version, focusing on the parts where the bias is most pronounced. By distilling this information into a special adapter, D2D amplifies the bias signal, making it easier to spot in the model's outputs. This approach not only helps in detecting hidden biases but also provides a theoretical framework to understand why it works.
Compared to previous methods, D2D offers a practical solution for auditing language models in real-world applications. It allows builders to identify and address biases that could influence user decisions, ensuring that the models they deploy are fairer and more transparent.
The method introduces a novel approach to detecting hidden biases in language models.
The claims are supported by empirical observations and a theoretical framework, though more extensive validation could strengthen them.
Deep reliability assessment
The methodology supports detecting hidden biases by amplifying them through a cartridge distillation process, but it may overclaim by not fully addressing the variability of bias topics in real-world applications. The theoretical framework is supported by empirical observations but may not cover all potential bias scenarios.
Reproducibility
Yes, the paper provides a GitHub repository for the code: https://github.com/abhinav-chinta/Distill2Detect.
Key figure
Figure 1 illustrates the Distill to Detect (D2D) process, showing how a biased model's response is distilled into a cartridge to amplify and reveal hidden biases for detection.