Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool

Imagine you're developing a machine learning system that needs to adapt quickly to new models and frameworks. The challenge is that as ML evolves, the underlying assumptions of your performance models often become outdated, leading to a constant cycle of refactoring and technical debt. This is particularly problematic because traditional coding practices can be cumbersome and slow, making it hard to keep up with rapid advancements in the field. This situation is what's called tech debt, where the cost of maintaining and updating code becomes higher than starting anew.

To address this, the authors propose SMART, a performance-modeling library that relies on natural-language design documents instead of conventional code. The idea is that these documents serve as self-contained guides that coding agents can use to regenerate the necessary code whenever updates are needed. By focusing on step-by-step examples and a minimal operator intermediate representation, SMART allows for reliable regeneration of implementations that closely match established reference models. This means that rather than wrestling with outdated code, builders can focus on maintaining clear, understandable documentation that evolves alongside their systems, making the design process more efficient and less error-prone.

The approach of using natural-language design docs for performance modeling is a significant shift from traditional coding practices.

The method demonstrates reliable regeneration of implementations that match hand-audited models, indicating solid performance.