An optimization-based equilibrium measure describes non-equilibrium steady state dynamics: application to edge of chaos
CoRR(2024)
摘要
Understanding neural dynamics is a central topic in machine learning,
non-linear physics and neuroscience. However, the dynamics is non-linear,
stochastic and particularly non-gradient, i.e., the driving force can not be
written as gradient of a potential. These features make analytic studies very
challenging. The common tool is to use path integral approach or dynamical
mean-field theory, but the drawback is one has to solve the
integro-differential or dynamical mean-field equations, which is
computationally expensive and has no closed form solutions in general. From the
aspect of associated Fokker-Planck equation, the steady state solution is
generally unknown. Here, we treat searching for the steady state as an
optimization problem, and construct an approximate potential closely related to
the speed of the dynamics, and find that searching for the ground state of this
potential is equivalent to running a stochastic gradient dynamics. The
resultant stationary state follows exactly the canonical Boltzmann measure.
Within this framework, the quenched disorder intrinsic in the neural networks
can be averaged out by applying the replica method. Our theory reproduces the
well-known result of edge-of-chaos, and further the order parameters
characterizing the continuous transition are derived, and different scaling
behavior with respect to inverse temperature in both sides of the transition is
also revealed. Our method opens the door to analytically study the steady state
landscape of the deterministic or stochastic high dimensional dynamics.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要