SHE: Trajectory-driven Safety Harness Evolution for LLM Agents
Wanying Qu, Qinghua Mao, Yu Li, Jiyao Liu, Xin Zhang, Dadi Guo, Yanxu Zhu, Qingyu Liu, Leitao Yuan, Xi Lin, Shanfeng Zhu, Yanwei Fu, Jing Shao, Xia Hu, Dongrui Liu
Read on arXiv →
Key claim
SHE enhances safety through localized evolution of harness components.
In plain English
Imagine you're building a large language model (LLM) agent that needs to operate safely in dynamic environments, like assisting users with various tasks while adapting to new information and risks. Currently, many safety mechanisms treat the agent's operational framework as a static setup, which can lead to failures when unexpected situations arise. For instance, if the agent encounters a new type of harmful input, it might not know how to respond appropriately, leading to what's called safety drift — where the agent's behavior becomes less safe over time due to unaddressed risks. This can manifest as incorrect tool usage or failure to maintain context, ultimately compromising user trust and safety. To address these issues, the authors propose a framework called Safety Harness Evolution (SHE), which breaks down the agent's operational harness into distinct components, each with specific safety responsibilities. This decomposition allows for localized improvements and a feedback loop that learns from failures during operation. By analyzing rollout trajectories, SHE can refine safety boundaries and adapt the harness to better handle emerging risks. The results show that SHE significantly reduces safety risks while maintaining or even improving the agent's utility, demonstrating its effectiveness in evolving safety mechanisms compared to static approaches.
The framework introduces a novel approach to evolving safety mechanisms in LLMs.
Experiments show significant improvements in safety metrics, though more baselines could strengthen claims.
Deep reliability assessment
The methodology supports evolving safety boundaries through trajectory analysis and artifact-specific updates, but the generalization to unseen risks and transfer across models may be overclaimed without extensive validation.
Reproducibility
Yes, the paper mentions a GitHub repository for the code.
Key figure
Figure 1 illustrates the motivation for Safety Harness Evolution (SHE) by showing how decoupling the safety harness into editable artifacts enables targeted diagnosis and localized evolution.