GRASP: Plan-Guided Graph Retrieval with Adaptive Fusion and Reranking on Semi-Structured Knowledge Bases
Yicheng Tao, Yiqun Wang, Xiangchen Song, Xin Luo, Kai Liu, Jie Liu
Read on arXiv →
Key claim
GRASP improves Hit@1 from 62.0 to 73.9.
The GRASP framework enhances retrieval from semi-structured knowledge bases by integrating graph retrieval with dense retrieval and reranking. It achieves a notable improvement in Hit@1 scores, demonstrating its effectiveness across various benchmarks. This advancement could greatly benefit applications in product and academic search.
In plain English
The authors developed a new framework called GRASP that improves how we retrieve information from semi-structured knowledge bases, which are databases that organize data in a graph format with entities and relationships. Unlike previous methods that either only used the graph for expanding queries or combined text and structure in a simplistic way, GRASP uses a three-step process that includes planning how to navigate the graph, merging this with a dense retrieval method, and then refining the results through a reranking step. This approach led to a significant increase in retrieval accuracy, as shown by the improvement in Hit@1 scores from 62.0 to 73.9 across various benchmarks. For builders, this means that applications like product searches or academic paper searches can become much more effective, providing users with more relevant results. Understanding and implementing GRASP could enhance the performance of systems that rely on complex data relationships.
GRASP introduces a novel three-stage retrieval framework that significantly improves upon existing hybrid retrieval systems.
The paper provides strong empirical results across multiple benchmarks and includes ablation studies to support its claims.
Deep reliability assessment
The methodology supports the integration of graph and dense retrieval methods for improved performance on semi-structured knowledge bases, but the generalization to other types of knowledge bases is not tested.
Reproducibility
No open source code or dataset is mentioned in the paper.
Discussion questions
- How does the reliance on a strong instruction-tuned LLM affect the generalizability of GRASP to other domains?
- What are the practical implications of GRASP's performance improvements for real-world applications like e-commerce or academic search?
- What specific scenarios or datasets would challenge the effectiveness of GRASP's fusion strategy?
Key figure
Figure 1 provides an overview of the GRASP framework with a concrete example from the Amazon dataset.