A Continuous-Time Perspective on Global Acceleration for Monotone Equation Problems
arxiv(2022)
摘要
We propose a new framework to design and analyze accelerated methods that
solve general monotone equation (ME) problems F(x)=0. Traditional approaches
include generalized steepest descent methods and inexact Newton-type methods.
If F is uniformly monotone and twice differentiable, these methods achieve
local convergence rates while the latter methods are globally convergent thanks
to line search and hyperplane projection. However, a global rate is unknown for
these methods. The variational inequality methods can be applied to yield a
global rate that is expressed in terms of F(x) but these results are
restricted to first-order methods and a Lipschitz continuous operator. It has
not been clear how to obtain global acceleration using high-order Lipschitz
continuity. This paper takes a continuous-time perspective where accelerated
methods are viewed as the discretization of dynamical systems. Our contribution
is to propose accelerated rescaled gradient systems and prove that they are
equivalent to closed-loop control systems. Based on this connection, we
establish the properties of solution trajectories. Moreover, we provide a
unified algorithmic framework obtained from discretization of our system, which
together with two approximation subroutines yields both existing high-order
methods and new first-order methods. We prove that the p^th-order method
achieves a global rate of O(k^-p/2) in terms of F(x) if F is
p^th-order Lipschitz continuous and the first-order method achieves the
same rate if F is p^th-order strongly Lipschitz continuous. If F is
strongly monotone, the restarted versions achieve local convergence with order
p when p ≥ 2. Our discrete-time analysis is largely motivated by the
continuous-time analysis and demonstrates the fundamental role that rescaled
gradients play in global acceleration for solving ME problems.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要