Interpretable AI with Local Distillation

Imagine you're working on a healthcare AI that predicts patient outcomes based on complex data like gene expressions. While modern AI models can make accurate predictions, they often act like black boxes, leaving healthcare professionals unsure about how decisions are made. This lack of transparency can be problematic, especially in high-stakes situations where understanding the reasoning behind a prediction is crucial. Current methods often fail to provide insights into the model's decision-making process, which is what's called interpretability failure. This can lead to mistrust in the model's predictions and hinder its adoption in critical applications. To address this, the authors propose a method called local distillation, which combines the strengths of a complex black-box model (the teacher) with a simpler, interpretable linear model (the student). The teacher helps the student by focusing on similar predictions to define what 'local' means, and it provides guidance at each query point. By adding a bit of randomness to the training process, the method assesses which features are consistently important, allowing for a clearer understanding of the model's behavior. This approach not only matches the accuracy of the black-box model but also produces a sparse linear model that reveals different patient subgroups based on their gene expressions, something that a global model might miss. Compared to previous work, local distillation offers a way to maintain high accuracy while significantly improving interpretability, making it a valuable tool for builders in fields where understanding model decisions is essential.

The approach introduces local distillation, a novel method for enhancing interpretability in AI models.

The method is validated across 17 benchmark datasets, demonstrating solid performance and stability.