Multi-level semantic enhancement based on self-distillation BERT for Chinese named entity recognition

As an important foundational task in the field of natural language processing, the Chinese named entity recognition (NER) task has received widespread attention in recent years. Self-distillation plays a role in exploring the potential of the knowledge carried by internal parameters in the BERT NER model, but few studies have noticed the impact of different granularity semantic information during the distillation process. In this paper, we propose a multi-level semantic enhancement approach based on self-distillation BERT for Chinese named entity recognition. We first design a feasible data augmentation method to improve the training quality for handling complex entity compositions, then construct a boundary smoothing module to achieve the model’s moderate learning on entity boundaries. Besides, we utilize the distillation reweighting method to let the model acquire balanced entity and context knowledge. Experimental results on two Chinese named entity recognition benchmark datasets Weibo and Resume have 72.09% and 96.93% F1 scores, respectively. Compared to three different basic distillation BERT models, our model can also produce better results. The source code is available at https://github.com/lookmedandan/MSE.