Robust Second-Order Nonconvex Optimization and Its Application to Low Rank Matrix Sensing
NeurIPS(2024)
摘要
Finding an approximate second-order stationary point (SOSP) is a well-studied
and fundamental problem in stochastic nonconvex optimization with many
applications in machine learning. However, this problem is poorly understood in
the presence of outliers, limiting the use of existing nonconvex algorithms in
adversarial settings.
In this paper, we study the problem of finding SOSPs in the strong
contamination model, where a constant fraction of datapoints are arbitrarily
corrupted. We introduce a general framework for efficiently finding an
approximate SOSP with dimension-independent accuracy guarantees, using
O(D^2/ϵ) samples where D is the ambient dimension
and ϵ is the fraction of corrupted datapoints.
As a concrete application of our framework, we apply it to the problem of low
rank matrix sensing, developing efficient and provably robust algorithms that
can tolerate corruptions in both the sensing matrices and the measurements. In
addition, we establish a Statistical Query lower bound providing evidence that
the quadratic dependence on D in the sample complexity is necessary for
computationally efficient algorithms.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要