The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric
Sheng-Yu Wang, Yotam Nitzan, Aaron Hertzmann, Jun-Yan Zhu, Eli Shechtman, Alexei A. Efros, Richard Zhang
Read on arXiv →Key claim
TPIPS metric captures nuanced visual similarity better than existing methods.
In plain English
Human visual similarity judgments depend on context, but current metrics oversimplify this into a single value. This paper introduces a dataset of human similarity judgments over image triplets, annotated for various aspects of similarity. It develops a new metric, TPIPS, that captures these nuances and shows improved alignment with human perception. Builders might find TPIPS useful for enhancing retrieval systems and evaluating generative models more effectively.
Introduces a new metric for visual similarity that captures multiple aspects.
Benchmarks against human judgments and shows generalization beyond training data.