Evaluating a Radius-based Pipeline for Question Answering over Cultural (CIDOC-CRM based) Knowledge Graphs

CIDOC-CRM is an event-based international standard for cultural documentation that has been widely used for offering semantic interoperability in the Cultural Heritage (CH) domain. Although there are several Knowledge Graphs (KGs) expressed by using CIDOC-CRM, the task of Question Answering (QA) has not been studied over such graphs. For this reason, in this paper we propose and evaluate a Radius-based QA pipeline over CIDOC-CRM KGs for single-entity factoid questions. In particular, we propose a generic QA pipeline that comprises several models and methods, including a keyword search model for recognizing the entity of the question (and linking it to the KG), methods that are based on path expansion for constructing subgraphs of different radius (i.e., path lengths) starting from the recognized entity, i.e., for being used as a context, and pre-trained neural models (based on BERT) for answering the question using the mentioned context. Moreover, since there are no available benchmarks over CIDOC-CRM KGs, we construct (by using a real KG) an evaluation benchmark having 10,000 questions, i.e., 5,000 single-entity factoid, 2,500 comparative and 2,500 confirmation questions. For evaluating the QA pipeline, we use the 5,000 single-entity factoid questions. Concerning the results, the QA pipeline achieves satisfactory results both in the entity recognition step (78% accuracy) and in the QA process (51% F1 score).