Fine-Grained Visual Comparisons with Local Learning

Given two images, we want to predict which exhibits a particular visual attribute more than the other -- even when the two images are quite similar. Existing relative attribute methods rely on global ranking functions, yet rarely will the visual cues relevant to a comparison be constant for all data, nor will humans' perception of the attribute necessarily permit a global ordering. To address these issues, we propose a local learning approach for fine-grained visual comparisons. Given a novel pair of images, we learn a local ranking model on the fly, using only analogous training comparisons. We show how to identify these analogous pairs using learned metrics. With results on three challenging datasets -- including a large newly curated dataset for fine-grained comparisons -- our method outperforms state-of-the-art methods for relative attribute prediction.