Enhance Robustness of Language Models Against Variation Attack through Graph Integration
International Conference on Computational Linguistics(2024)
摘要
The widespread use of pre-trained language models (PLMs) in natural language
processing (NLP) has greatly improved performance outcomes. However, these
models' vulnerability to adversarial attacks (e.g., camouflaged hints from drug
dealers), particularly in the Chinese language with its rich character
diversity/variation and complex structures, hatches vital apprehension. In this
study, we propose a novel method, CHinese vAriatioN Graph Enhancement (CHANGE),
to increase the robustness of PLMs against character variation attacks in
Chinese content. CHANGE presents a novel approach for incorporating a Chinese
character variation graph into the PLMs. Through designing different
supplementary tasks utilizing the graph structure, CHANGE essentially enhances
PLMs' interpretation of adversarially manipulated text. Experiments conducted
in a multitude of NLP tasks show that CHANGE outperforms current language
models in combating against adversarial attacks and serves as a valuable
contribution to robust language model research. These findings contribute to
the groundwork on robust language models and highlight the substantial
potential of graph-guided pre-training strategies for real-world applications.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要