Generating Pseudo Connectives with MLMs for Implicit Discourse Relation Recognition.

Due to the lack of connectives, the recognition of implicit discourse relations faces a big challenge. An early attempt overcomes this difficulty by predicting connectives with the use of the statistical language model. Recent years have witnessed the great success of masked language models (MLM). Then a new problem naturally arises, i.e., how can connectives benefit implicit discourse relation classification from such models? In this paper, we address this problem by developing a novel framework to generate the pseudo connectives using the pre-trained MLM. The key idea is to treat the absent connectives as missing words between two arguments and produce the pseudo connective from its contexts by fine-tuning MLM on the classification task. Moreover, we leverage the real connectives in explicit discourse relations to supervise the generation of pseudo connectives. Extensive experiments show that our model achieves the state-of-the-art performance on the PDTB benchmark.