Cascaded Compressed Sensing Networks

For deep networks with complex nonlinearity, the structure analysis and design remain challenging. In the letter, we propose to understand and build deep networks as a cascade of compressed sensing. Each compressed sensing module consists of two layers, corresponding to the two data transforms involved in compressed sensing, namely the sparse transform over dictionaries and the measurement over random projection. The two transforms can be viewed as a process of feature generation and extraction, thus amenable to network construction. The proposed construction has two advantages. First, it can be analyzed with compressed sensing theory. Second, it enables networks to be learned layer by layer by back-propagating the target via compressed sensing, and empirically the layerwise learning method tends to outperform the conventional error back-propagation method in the presence of noise.