A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarios
arxiv(2024)
摘要
We evaluate the robustness of several large language models on multiple
datasets. Robustness here refers to the relative insensitivity of the model's
answers to meaning-preserving variants of their input. Benchmark datasets are
constructed by introducing naturally-occurring, non-malicious perturbations, or
by generating semantically equivalent paraphrases of input questions or
statements. We further propose a novel metric for assessing a model robustness,
and demonstrate its benefits in the non-adversarial scenario by empirical
evaluation of several models on the created datasets.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要