An Empirical Study of Uncertainty Estimation Techniques for Detecting Drift in Data Streams.
CoRR(2023)
摘要
In safety-critical domains such as autonomous driving and medical diagnosis,
the reliability of machine learning models is crucial. One significant
challenge to reliability is concept drift, which can cause model deterioration
over time. Traditionally, drift detectors rely on true labels, which are often
scarce and costly. This study conducts a comprehensive empirical evaluation of
using uncertainty values as substitutes for error rates in detecting drifts,
aiming to alleviate the reliance on labeled post-deployment data. We examine
five uncertainty estimation methods in conjunction with the ADWIN detector
across seven real-world datasets. Our results reveal that while the SWAG method
exhibits superior calibration, the overall accuracy in detecting drifts is not
notably impacted by the choice of uncertainty estimation method, with even the
most basic method demonstrating competitive performance. These findings offer
valuable insights into the practical applicability of uncertainty-based drift
detection in real-world, safety-critical applications.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要