Towards Understanding and Answering Comparative Questions

ABSTRACTIn this paper, we analyze comparative questions and answers. At least 3%~of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000~English questions from existing datasets that we label as comparative or not. In the 3,500~comparative questions, we label the comparison objects, aspects, and predicates. For 950~questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall~71% of the comparative questions with a perfect precision of~1.0, recall~92% of subjective comparative questions with a precision of~0.98, and identify the comparison objects and aspects with an F1 of~0.93 and~0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of~0.63.