Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data
arXiv (Cornell University)(2022)
摘要
Implementing effective control mechanisms to ensure the proper functioning
and security of deployed NLP models, from translation to chatbots, is
essential. A key ingredient to ensure safe system behaviour is
Out-Of-Distribution (OOD) detection, which aims to detect whether an input
sample is statistically far from the training distribution. Although OOD
detection is a widely covered topic in classification tasks, most methods rely
on hidden features output by the encoder. In this work, we focus on leveraging
soft-probabilities in a black-box framework, i.e. we can access the
soft-predictions but not the internal states of the model. Our contributions
include: (i) RAINPROOF a Relative informAItioN Projection OOD detection
framework; and (ii) a more operational evaluation setting for OOD detection.
Surprisingly, we find that OOD detection is not necessarily aligned with
task-specific measures. The OOD detector may filter out samples well processed
by the model and keep samples that are not, leading to weaker performance. Our
results show that RAINPROOF provides OOD detection methods more aligned with
task-specific performance metrics than traditional OOD detectors.
更多查看译文
关键词
shield text generators,data,umbrella,out-of-distribution
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要