Semantic Parsing for Single-Relation Question Answering

ACL, pp. 643-648, 2014.

Cited by: 275|Bibtex|Views44|Links
EI
Keywords:
knowledge basequestion answeringdifferent recallsemantic similaritysingle relation questionMore(9+)
Wei bo:
We propose a semantic parsing framework for single-relation questions

Abstract:

We develop a semantic parsing framework based on semantic similarity for open domain question answering (QA). We focus on single-relation questions and decompose each question into an entity mention and a relation pattern. Using convolutional neural network models, we measure the similarity of entity mentions with entities in the knowledg...More

Code:

Data:

0
Introduction
  • Open-domain question answering (QA) is an important and yet challenging problem that remains largely unsolved.
  • The authors focus on answering single-relation factual questions, which are the most common type of question observed in various community QA sites (Fader et al, 2013), as well as in search query logs
  • The authors assumed such questions are answerable by issuing a singlerelation query that consists of the relation and an argument entity, against a knowledge base (KB).
  • That is to say that the problem of mapping from a question to a particular relation and entity in the KB is non-trivial
Highlights
  • Open-domain question answering (QA) is an important and yet challenging problem that remains largely unsolved
  • We focus on answering single-relation factual questions, which are the most common type of question observed in various community question answering sites (Fader et al, 2013), as well as in search query logs
  • We propose a semantic parsing framework tailored to single-relation questions
  • Mine the probabilities of such mappings, we propose using a semantic similarity model based on convolutional neural networks, which is the technical focus in this paper
  • Due to the variety of entity mentions in the real world, the parallel corpus derived from the WikiAnswers data and ReVerb knowledge base may not contain enough data to train a robust entity linking model
Methods
  • In order to provide a fair comparison to previous work, the authors experimented with the approach using the PARALAX dataset (Fader et al, 2013), which consists of paraphrases of questions mined from WikiAnswers and answer triples from ReVerb.
  • The authors first scanned the original training corpus to see if there was an exact surface form match of the entity.
  • The authors derived about 1.2 million pairs of patterns and relations
  • The authors applied these patterns to all the 1.8 million training questions, which helped discover 160 thousand new mentions that did not have the exact surface form matches to the entities
Results
  • The system first enumerated all possible decompositions of the mentions and patterns, as described earlier.
  • The authors computed the similarity scores between the pattern and all relations in the KB and retained 150 top-scoring relation candidates.
  • The system checked all triples in the KB that had this relation and computed the similarity score between the mention and corresponding argument entity.
  • The top answer triple was used to compute the precision and recall of the system when reporting the system performance.
  • By limiting the systems to output only answer triples with scores higher than a predefined threshold, the authors could control the trade-off between recall and precision and plot the precision–recall curve
Conclusion
  • The authors propose a semantic parsing framework for single-relation questions. Compared to the existing work, the key insight is to match relation patterns and entity mentions using a semantic similarity function rather than lexical rules.
  • Due to the variety of entity mentions in the real world, the parallel corpus derived from the WikiAnswers data and ReVerb KB may not contain enough data to train a robust entity linking model.
  • Replacing this component with a dedicated entity linking system could improve the performance and reduce the number of pattern/mention candidates when processing each question.
  • The authors would like to extend the method to other more structured KBs, such as Freebase, and to explore approaches to extend the system to handle multi-relation questions
Summary
  • Introduction:

    Open-domain question answering (QA) is an important and yet challenging problem that remains largely unsolved.
  • The authors focus on answering single-relation factual questions, which are the most common type of question observed in various community QA sites (Fader et al, 2013), as well as in search query logs
  • The authors assumed such questions are answerable by issuing a singlerelation query that consists of the relation and an argument entity, against a knowledge base (KB).
  • That is to say that the problem of mapping from a question to a particular relation and entity in the KB is non-trivial
  • Methods:

    In order to provide a fair comparison to previous work, the authors experimented with the approach using the PARALAX dataset (Fader et al, 2013), which consists of paraphrases of questions mined from WikiAnswers and answer triples from ReVerb.
  • The authors first scanned the original training corpus to see if there was an exact surface form match of the entity.
  • The authors derived about 1.2 million pairs of patterns and relations
  • The authors applied these patterns to all the 1.8 million training questions, which helped discover 160 thousand new mentions that did not have the exact surface form matches to the entities
  • Results:

    The system first enumerated all possible decompositions of the mentions and patterns, as described earlier.
  • The authors computed the similarity scores between the pattern and all relations in the KB and retained 150 top-scoring relation candidates.
  • The system checked all triples in the KB that had this relation and computed the similarity score between the mention and corresponding argument entity.
  • The top answer triple was used to compute the precision and recall of the system when reporting the system performance.
  • By limiting the systems to output only answer triples with scores higher than a predefined threshold, the authors could control the trade-off between recall and precision and plot the precision–recall curve
  • Conclusion:

    The authors propose a semantic parsing framework for single-relation questions. Compared to the existing work, the key insight is to match relation patterns and entity mentions using a semantic similarity function rather than lexical rules.
  • Due to the variety of entity mentions in the real world, the parallel corpus derived from the WikiAnswers data and ReVerb KB may not contain enough data to train a robust entity linking model.
  • Replacing this component with a dedicated entity linking system could improve the performance and reduce the number of pattern/mention candidates when processing each question.
  • The authors would like to extend the method to other more structured KBs, such as Freebase, and to explore approaches to extend the system to handle multi-relation questions
Tables
  • Table1: Performance of two variations of our systems, compared with the PARALEX system
Download tables as Excel
Related work
  • Semantic parsing of questions, which maps natural language questions to database queries, is a critical component for KB-supported QA. An early example of this research is the semantic parser for answering geography-related questions, learned using inductive logic programming (Zelle and Mooney, 1996). Research in this line originally used small, domain-specific databases, such as GeoQuery (Tang and Mooney, 2001; Liang et

    Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Short Papers), pages 643–648, Baltimore, Maryland, USA, June 23-25 2014. c 2014 Association for Computational Linguistics al., 2013). Very recently, researchers have started developing semantic parsers for large, generaldomain knowledge bases like Freebase and DBpedia (Cai and Yates, 2013; Berant et al, 2013; Kwiatkowski et al, 2013). Despite significant progress, the problem remains challenging. Most methods have not yet been scaled to large KBs that can support general open-domain QA. In contrast, Fader et al (2013) proposed the PARALEX system, which targets answering single-relation questions using an automatically created knowledge base, ReVerb (Fader et al, 2011). By applying simple seed templates to the KB and by leveraging community-authored paraphrases of questions from WikiAnswers, they successfully demonstrated a high-quality lexicon of patternmatching rules can be learned for this restricted form of semantic parsing.
Reference
  • Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533–1544, Seattle, Washington, USA, October. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. the Journal of machine Learning research, 3:993–1022.
    Google ScholarLocate open access versionFindings
  • Qingqing Cai and Alexander Yates. 201Largescale semantic parsing via schema matching and lexicon extension. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 423–433, Sofia, Bulgaria, August. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research.
    Google ScholarLocate open access versionFindings
  • Scott Deerwester, Susan Dumais, Thomas Landauer, George Furnas, and Richard Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6).
    Google ScholarLocate open access versionFindings
  • Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In Proceedings of the Conference of Empirical Methods in Natural Language Processing (EMNLP ’11), Edinburgh, Scotland, UK, July 2731.
    Google ScholarLocate open access versionFindings
  • Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2013. Paraphrase-driven learning for open question answering. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1608–1618, Sofia, Bulgaria, August. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management, pages 2333–233ACM.
    Google ScholarLocate open access versionFindings
  • Tom Kwiatkowski, Eunsol Choi, Yoav Artzi, and Luke Zettlemoyer. 2013. Scaling semantic parsers with on-the-fly ontology matching. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1545–1556, Seattle, Washington, USA, October. Association for Computational Linguistics.
    Google ScholarLocate open access versionFindings
  • Percy Liang, Michael I Jordan, and Dan Klein. 2013. Learning dependency-based compositional semantics. Computational Linguistics, 39(2):389–446.
    Google ScholarLocate open access versionFindings
  • Gregoire Mesnil, Xiaodong He, Li Deng, and Yoshua Bengio. 2013. Investigation of recurrent-neuralnetwork architectures and learning methods for spoken language understanding. In Interspeech.
    Google ScholarFindings
  • Ruslan Salakhutdinov and Geoffrey Hinton. 2009. Semantic hashing. International Journal of Approximate Reasoning, 50(7):969–978.
    Google ScholarLocate open access versionFindings
  • Gerard Salton and Michael J. McGill. 1983. Introduction to Modern Information Retrieval. McGraw Hill.
    Google ScholarFindings
  • Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Gregoire Mesnil. 2014a. A convolutional latent semantic model for web search. Technical Report MSR-TR-2014-55, Microsoft Research.
    Google ScholarFindings
  • Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Gregoire Mesnil. 2014b. Learning semantic representations using convolutional neural networks for web search. In Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, pages 373–374.
    Google ScholarLocate open access versionFindings
  • Lappoon Tang and Raymond Mooney. 2001. Using multiple clause constructors in inductive logic programming for semantic parsing. In Machine Learning: ECML 2001, pages 466–477. Springer.
    Google ScholarLocate open access versionFindings
  • Gokhan Tur, Li Deng, Dilek Hakkani-Tur, and Xiaodong He. 2012. Towards deeper understanding: deep convex networks for semantic utterance classification. In Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, pages 5045–5048. IEEE.
    Google ScholarLocate open access versionFindings
  • John Zelle and Raymond Mooney. 1996. Learning to parse database queries using inductive logic programming. In Proceedings of the National Conference on Artificial Intelligence, pages 1050–1055.
    Google ScholarLocate open access versionFindings
Your rating :
0

 

Tags
Comments