SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization

Imagine you're building a system that needs to interpret complex data from large language models, like understanding the features they extract. Currently, the process of explaining these features often relies on external observations, which can lead to shallow insights and inefficiencies when trying to gather evidence at scale. This reliance can result in what's called superficial explanations, where the true meaning behind the model's behavior remains unclear. As a builder, you want a more direct way to understand and communicate these features without getting bogged down in cumbersome observation methods.

The authors propose a solution with SAEVerbalizer, which integrates decoder directions from sparse autoencoders directly into the representations of a language model. This framework fine-tunes the model's downstream layers to produce natural-language explanations of the features, allowing for a more intuitive understanding of what the model is doing. The results show that this verbalization capability not only generalizes to unseen features but also adapts across different sparse autoencoder dictionaries and models. This means that as a builder, you can leverage this method to create systems that explain their reasoning more transparently and effectively, ultimately leading to better user trust and understanding.

The approach introduces a novel method for generating natural-language explanations from sparse autoencoders.

The experiments demonstrate generalization and transferability, though more extensive baselines could strengthen claims.