Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Prediction

Imagine you're developing a clinical AI that needs to accurately predict patient outcomes based on hospital notes. The challenge is that these notes often contain irrelevant artifacts like templates and boilerplate text that can mislead the model, leading to poor performance when deployed in real-world settings. This issue, known as deployment shift, means that while models may perform well in controlled environments, they struggle to generalize to actual patient data. This is what's called overfitting to artifacts, where the model learns to rely on these misleading cues rather than the true clinical information. To address this, CAST (Concept-guided Artifact Suppression Tuning) offers a framework that uses Sparse Autoencoders to identify and suppress these artifacts during training. By focusing on human-auditable features and employing a pipeline that interprets model decisions, CAST not only improves prediction accuracy but also provides a clear audit trail of the clinical concepts involved. This means that when the model makes a prediction, you can trace back which concepts were influential and which artifacts were intentionally ignored. Compared to traditional fine-tuning methods, CAST not only enhances performance on tasks like mortality prediction but also ensures that the model's decisions are interpretable and reliable in real clinical settings.

CAST introduces a novel approach to suppressing artifacts in clinical text classification.

The evaluation on MIMIC-IV shows competitive results against strong baselines with clear methodology.