Evidence-Backed Video Question Answering
Shijie Wang, Honglu Zhou, Ziyang Wang, Ran Xu, Caiming Xiong, Silvio Savarese, Chen Sun, Juan Carlos Niebles
Read on arXiv →
Key claim
E-VQA improves explainability in video question answering.
In plain English
Video LLMs currently provide answers without clear visual grounding, making it hard to verify their responses. Existing methods for explainability are insufficient for capturing complex video dynamics. This paper introduces E-VQA, which requires models to output both answers and detailed visual evidence, along with a new benchmark for evaluation. Builders might care because this approach enhances the interpretability of video models, potentially leading to more reliable applications in real-world scenarios.
Introduces a new task and benchmark for explainable video QA.
Evaluates state-of-the-art models with a new, human-verified dataset.
Deep reliability assessment
The methodology supports the claim that E-VQA can improve explainability by providing spatio-temporal evidence, but the reliance on automated pipelines for dataset creation may introduce biases not fully addressed in the paper.
Reproducibility
Yes, the paper provides open source code and data at https://github.com/SalesforceAIResearch/EVQA.
Key figure
Figure 1 illustrates the E-VQA framework, showing how models provide textual answers while grounding reasoning in spatio-temporal evidence, including video segments and segmentation masks.