Using Natural Language Explanations to Improve Robustness of In-context Learning
arxiv(2023)
摘要
Recent studies demonstrated that large language models (LLMs) can excel in
many tasks via in-context learning (ICL). However, recent works show that
ICL-prompted models tend to produce inaccurate results when presented with
adversarial inputs. In this work, we investigate whether augmenting ICL with
natural language explanations (NLEs) improves the robustness of LLMs on
adversarial datasets covering natural language inference and paraphrasing
identification. We prompt LLMs with a small set of human-generated NLEs to
produce further NLEs, yielding more accurate results than both a zero-shot-ICL
setting and using only human-generated NLEs. Our results on five popular LLMs
(GPT3.5-turbo, Llama2, Vicuna, Zephyr, and Mistral) show that our approach
yields over 6
datasets: HANS, ISCS, NaN, ST, PICD, PISP, ANLI, and PAWS. Furthermore,
previous studies have demonstrated that prompt selection strategies
significantly enhance ICL on in-distribution test sets. However, our findings
reveal that these strategies do not match the efficacy of our approach for
robustness evaluations, resulting in an accuracy drop of 8
proposed approach.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要