Fast Adversarial Training against Textual Adversarial Attacks
CoRR(2024)
摘要
Many adversarial defense methods have been proposed to enhance the
adversarial robustness of natural language processing models. However, most of
them introduce additional pre-set linguistic knowledge and assume that the
synonym candidates used by attackers are accessible, which is an ideal
assumption. We delve into adversarial training in the embedding space and
propose a Fast Adversarial Training (FAT) method to improve the model
robustness in the synonym-unaware scenario from the perspective of single-step
perturbation generation and perturbation initialization. Based on the
observation that the adversarial perturbations crafted by single-step and
multi-step gradient ascent are similar, FAT uses single-step gradient ascent to
craft adversarial examples in the embedding space to expedite the training
process. Based on the observation that the perturbations generated on the
identical training sample in successive epochs are similar, FAT fully utilizes
historical information when initializing the perturbation. Extensive
experiments demonstrate that FAT significantly boosts the robustness of BERT
models in the synonym-unaware scenario, and outperforms the defense baselines
under various attacks with character-level and word-level modifications.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要