From Alexnet to Transformers: Measuring the Non-linearity of Deep Neural Networks with Affine Optimal Transport
CoRR(2023)
摘要
In the last decade, we have witnessed the introduction of several novel deep
neural network (DNN) architectures exhibiting ever-increasing performance
across diverse tasks. Explaining the upward trend of their performance,
however, remains difficult as different DNN architectures of comparable depth
and width – common factors associated with their expressive power – may
exhibit a drastically different performance even when trained on the same
dataset. In this paper, we introduce the concept of the non-linearity signature
of DNN, the first theoretically sound solution for approximately measuring the
non-linearity of deep neural networks. Built upon a score derived from
closed-form optimal transport mappings, this signature provides a better
understanding of the inner workings of a wide range of DNN architectures and
learning paradigms, with a particular emphasis on the computer vision task. We
provide extensive experimental results that highlight the practical usefulness
of the proposed non-linearity signature and its potential for long-reaching
implications. The code for our work is available at
https://github.com/qbouniot/AffScoreDeep
更多查看译文
关键词
Recurrent Neural Networks,Function Approximation,Neural Network Architectures,Backpropagation Learning,Radial Basis Function Networks
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要