Towards Understanding and Answering Comparative Questions

authored by: Alexander Bondarenko, Yamen Ajjour, Valentin Dittmar, Niklas Homann, Pavel Braslavski, Matthias Hagen
Abstract: In this paper, we analyze comparative questions and answers. At least 3%∼of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000∼English questions from existing datasets that we label as comparative or not. In the 3,500∼comparative questions, we label the comparison objects, aspects, and predicates. For 950∼questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall∼71% of the comparative questions with a perfect precision of∼1.0, recall∼92% of subjective comparative questions with a precision of∼0.98, and identify the comparison objects and aspects with an F1 of∼0.93 and∼0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of∼0.63.
Organisation(s): Natural Language Processing Section
External Organisation(s): Martin Luther University Halle-Wittenberg
Ural Federal University (UrFU)
Type: Conference contribution
Pages: 66-74
No. of pages: 9
Publication date: 15.02.2022
Publication status: Published
Peer reviewed: Yes
ASJC Scopus subject areas: Computer Networks and Communications, Computer Science Applications, Software
Electronic version(s): https://doi.org/10.1145/3488560.3498534 (Access: Closed)

BibTeX

@inproceedings{58b677495d6b4253938af653c87f39e7,
title = "Towards Understanding and Answering Comparative Questions",
abstract = "In this paper, we analyze comparative questions and answers. At least 3%∼of the questions submitted to search engines are comparative; ranging from simple facts like {"}Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like {"}Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000∼English questions from existing datasets that we label as comparative or not. In the 3,500∼comparative questions, we label the comparison objects, aspects, and predicates. For 950∼questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall∼71% of the comparative questions with a perfect precision of∼1.0, recall∼92% of subjective comparative questions with a precision of∼0.98, and identify the comparison objects and aspects with an F1 of∼0.93 and∼0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of∼0.63.",
keywords = "Answer stance detection, Comparative questions, Comparison objects and aspects, Question intent understanding",
author = "Alexander Bondarenko and Yamen Ajjour and Valentin Dittmar and Niklas Homann and Pavel Braslavski and Matthias Hagen",
note = "Funding Information: This work has been partially supported by the DFG through the projects “ACQuA” and “ACQuA 2.0” (Answering Comparative Questions with Arguments; grants HA 5851/2-1 and HA 5851/2-2) as part of the priority program “RATIO: Robust Argumentation Machines” (SPP 1999). Pavel Braslavski acknowledges funding from the Ministry of Science and Higher Education of the Russian Federation (project 075-02-2021-1387). We are also thankful to Ekaterina Shir-shakova and Jonas Hirsch for their help with the data annotation and code development.",
year = "2022",
month = feb,
day = "15",
doi = "10.1145/3488560.3498534",
language = "English",
pages = "66--74",
editor = "Candan, {K. Selcuk} and Huan Liu and Leman Akoglu and Dong, {Xin Luna} and Jiliang Tang",
booktitle = "15th ACM International Conference on Web Search and Data Mining (WSDM 2022)",
publisher = "ACM DL",
}

Publication Details

Towards Understanding and Answering Comparative Questions

Funded by