Sequence-to-Sequence Generative Argumentative Dialogue Systems with Self-Attention

Natural language generation is an area of natural language processing with much room for improvement. Argument mining poses the problem of responding relevantly and argumentatively to input in addition to the challenge of generating coherent text. In this paper, we explore the possibility of creating a dialogue agent capable of holding argumentative discussion. The architecture we propose employs local self-attention to encode and decode individual responses and global attention to invoke information from past exchanges. We achieve 25% word accuracy rate on the Internet Argument Corpus v1, beating the Transformer and traditional RNNstyle sequence-to-sequence architectures. Additionally, our architecture can easily be generalized to other discussion-style scenarios, given appropriate data.