Selective Classification Under Distribution Shifts
arxiv(2024)
摘要
In selective classification (SC), a classifier abstains from making
predictions that are likely to be wrong to avoid excessive errors. To deploy
imperfect classifiers – imperfect either due to intrinsic statistical noise of
data or for robustness issue of the classifier or beyond – in high-stakes
scenarios, SC appears to be an attractive and necessary path to follow. Despite
decades of research in SC, most previous SC methods still focus on the ideal
statistical setting only, i.e., the data distribution at deployment is the same
as that of training, although practical data can come from the wild. To bridge
this gap, in this paper, we propose an SC framework that takes into account
distribution shifts, termed generalized selective classification, that covers
label-shifted (or out-of-distribution) and covariate-shifted samples, in
addition to typical in-distribution samples, the first of its kind in the SC
literature. We focus on non-training-based confidence-score functions for
generalized SC on deep learning (DL) classifiers and propose two novel
margin-based score functions. Through extensive analysis and experiments, we
show that our proposed score functions are more effective and reliable than the
existing ones for generalized SC on a variety of classification tasks and DL
classifiers.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要