Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

Imagine you're trying to improve a language model that needs to perform well in specific areas, like medical or legal texts, but getting expert feedback is too expensive or slow. Traditionally, people would rely on labeled data to train these models, but that can lead to problems when the model encounters new types of data it hasn't seen before. This is where things can go wrong: the model might not generalize well, or it might misinterpret the data because it hasn't been trained on it directly. This is known as performance degradation in out-of-domain scenarios.

To tackle these issues, the authors propose a new method called Neuron On-Policy Self-Distillation (Neuron-OPSD). Instead of needing labeled data, this approach uses the model's own outputs as a guide for training. It looks at the internal activations of the model's neurons to help select the right training data and build a context for learning. This way, the model can learn from its own predictions and improve its performance without needing external supervision.

The key takeaway is that Neuron-OPSD not only enhances performance in the areas it was trained on but also maintains its ability to generalize to new, unseen data. This is particularly useful in situations where getting human feedback is impractical, making it a valuable tool for developers working in specialized domains.

The approach introduces a new framework for self-distillation that leverages internal neuron activations, which is a significant extension of existing methods.

The claims are supported by benchmarks in specialized domains, though more extensive validation could strengthen the findings.