Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

Imagine you're building a machine learning model that needs to perform arithmetic operations like addition, but with a twist: it has to do this under certain constraints, like modular arithmetic. In practice, many models struggle with generalization after initially learning a task, especially when faced with variations in the data or structure. This is what's called overfitting, where the model performs well on training data but fails to adapt to new situations. The paper dives into this issue, particularly focusing on a model named Muon, which learns to add numbers quickly but then loses its ability to generalize effectively across different configurations and tasks. It highlights how certain optimizers, like AdamW, behave differently compared to Muon, especially when the model's parameters are frozen or altered. This leads to a deeper understanding of the representation-readout interface, where the model's learned knowledge fails to translate into effective performance on unseen data.

The authors propose that by freezing certain parts of the model, they can prevent this failure, which suggests that the way we structure and optimize our models can significantly impact their performance. They also explore how Fourier filtering can help distinguish between different types of failures in the model's architecture. This work shifts the focus from merely achieving high accuracy on training tasks to understanding the underlying mechanisms that lead to failure in real-world applications. For builders, this means that optimizing model architecture and training strategies is crucial for ensuring that models not only learn but also retain their ability to generalize effectively.

The paper introduces a new perspective on modular addition and optimization dynamics.

The findings are based on extensive experiments but may lack broader applicability.