UniMem: Complementary Episodic-to-Parametric Memory for Boundary-Agnostic Task Streams

Imagine you're building an AI that needs to learn from its experiences while adapting to new tasks over time. The challenge lies in balancing the need for quick learning from new information with the ability to retain and efficiently use established knowledge. Current systems often struggle with this; they either rely on external memory that can quickly adapt but is slow during execution, or they use fixed internal memory that can’t easily incorporate new tasks, leading to what's known as the stability-plasticity dilemma. This means they can either forget important patterns or become overloaded with irrelevant information. To address this, the authors propose a framework called UniMem, which mimics how the human brain manages memory. It uses learnable routing tokens to control memory access, allowing the system to dynamically decide whether to store new information in a fast-access episodic buffer or to consolidate it into a more stable parametric memory. This dual approach enables the AI to handle both novel and recurring tasks efficiently, without needing explicit task labels or risking uncontrolled memory growth. Compared to previous methods, UniMem shows significant improvements in handling long sequences of tasks, achieving an average gain of 4.0 EM points across different models. This means that for builders, UniMem could lead to more robust AI systems that can learn and adapt in real-world applications without the typical pitfalls of memory management.

UniMem introduces a novel approach to memory management in LLMs by combining episodic and parametric memory.

The experiments show consistent performance improvements across multiple backbone models, though details on baseline comparisons could be clearer.