Generative Skill Composition for LLM Agents

Imagine you're building a system that needs to perform a variety of complex tasks, like setting up environments or refactoring code. You could have a library of skills, but figuring out which skills to use, how many, and in what order can be really tricky. Right now, people either let the system see all the skills at once or use some kind of retrieval system to pick the best ones, but both methods can struggle with the complexity of task requirements. This is where things can go wrong: the system might pick the wrong skills or fail to execute them in the right order, leading to subpar performance. This issue is known as skill composition failure, where the interaction between skills isn't properly managed. What this paper introduces is a new way to handle this problem by formalizing it as structured skill composition. The idea is to predict a skill plan that not only selects which skills to use but also determines how many and the order in which they should be executed. The proposed method, SkillComposer, uses a clever decoding technique that allows these decisions to emerge together, capturing the dependencies between skills naturally. In practice, this means that SkillComposer can significantly improve task success rates, raising them by over 23% compared to a baseline that doesn't use skills. This is a big deal for anyone building systems that rely on complex task execution, as it streamlines the process and enhances overall performance.

The paper introduces a structured approach to skill composition that significantly extends existing methods for task execution in LLMs.

The evaluation on multiple production-grade coding agents provides solid evidence for the effectiveness of the proposed method.