RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experiments
Babak Rahmani, Sebastian Dziadzio, Joschka Strüber, Sergio Hernández-Gutiérrez, Matthias Bethge
Read on arXiv →
Key claim
Reconstructed policies improve competitive performance in games.
In plain English
Imagine you're trying to understand how a player makes decisions in a game, but all you have are their actions. Traditionally, researchers would guess the underlying decision-making process based on what they see, which can be tricky and often leads to incomplete or incorrect conclusions. This is known as the inverse problem — figuring out hidden mechanisms from observable behavior. The challenge is that without controlled experiments, it's hard to get clear insights into how decisions are made, leading to what we call limited interpretability of behavior.
Now, what if you could design specific scenarios to probe deeper into those decisions? This paper introduces a clever approach where a learner observes a player's actions and then creates custom opponents to elicit more informative responses. By doing this, the learner can reconstruct the underlying decision-making program as executable code. This method, called RevengeBench, uses a set of 75 policies generated by large language models across various game environments, allowing for a more structured way to analyze and understand player behavior.
The results are promising: the reconstructed policies not only provide insights into the decision-making process but also lead to measurable advantages in competitive play, particularly for weaker models that struggle to counter strong opponents. This advancement opens up new avenues for understanding how agents make decisions and could significantly enhance the interpretability of AI systems in complex environments.
The paper introduces a new benchmark for reconstructing decision programs from behavioral traces, which extends existing methods in a novel way.
The claims are supported by experiments across multiple game environments and LLMs, though further validation could strengthen the findings.
Deep reliability assessment
The methodology strongly supports claims about recovering executable, LLM-generated game policies under constrained observation/probing, measured by action-distance and some downstream tournament signal. It overreaches if read as evidence that the same approach will recover arbitrary real-world latent mechanisms, neural policies, or adaptive opponents, especially since probing gains are model-dependent and sometimes near zero.
Reproducibility
Partially reproducible from the provided text: the paper header advertises Leaderboard, Code, and Dataset links, and the benchmark is described as 75 policies across five CodeClash environments, but no exact repository or dataset URL is visible in the supplied excerpt.
Key figure
Figure 1 shows the learner first observing a hidden policy against sampled opponents, then writing custom probe opponents to elicit more informative behavior, alongside model-level recovery performance measured as fraction of initial action-distance closed.