SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints

Imagine you're building a personal assistant that not only helps users with tasks but also respects their privacy and choices. As these agents become more integrated into our lives, it’s crucial that they don’t just complete tasks but also uphold user sovereignty — meaning they should prioritize the user's interests without compromising their privacy or consent. However, current benchmarks often overlook this aspect, focusing mainly on task completion without considering how these agents might manipulate or mislead users. This is where the concept of sovereignty comes into play, highlighting the need for a more nuanced evaluation of personal agents.

The paper introduces SovereignPA-Bench, a new benchmark designed specifically to assess personal agents in terms of their ability to respect user sovereignty. It evaluates how well these agents navigate complex scenarios involving user preferences, privacy boundaries, and consent constraints. By separating what the agent can see from what evaluators can see, it provides a clearer picture of how these agents perform in real-world situations. The authors tested this benchmark across 120 scenarios and multiple model families, yielding a wealth of data that reveals how different approaches to agent design impact user sovereignty.

One key finding is that using a full-sovereign approach — which integrates memory, consent, and evidence considerations — significantly improves the agents' performance in maintaining user sovereignty compared to more traditional methods. This means that for builders creating personal agents, focusing on sovereignty not only enhances user trust but also leads to better overall performance in real-world applications.

The introduction of SovereignPA-Bench provides a significant new framework for evaluating personal agents with a focus on user sovereignty.

The evaluation is based on a comprehensive set of scenarios and metrics, ensuring solid experimental validation.