HISPOC: A High-Performance Irregular Activation Sparsity-Aware Point Cloud Network Accelerator

Voxel-based neural networks are promising algorithms to process 3D point cloud data. However, the voxel-based neural networks run inefficiently on GPUs for the unique calculation modes of 3D sparse convolution. In addition, the large irregular activation sparsity of 3D voxel-based neural networks can be utilized more to improve the performance of the hardware accelerator. This brief presents a high-performance 3D point cloud accelerator, namely HISPOC, which can effectively leverage irregular activation sparsity. Specifically, we propose a 3D bitmap-based search engine that significantly reduces the complexity of voxel searching. In addition, we propose a match-based input feature manager and a dichotomy-based output feature sparsity encoder to enhance computation efficiency and reduce RAM consumption based on the Zero-Value Compression algorithm. Finally, the average memory footprint of the feature is reduced to 54.98% . we implement the HISPOC on Xilinx Virtex VC707 board, achieving 3.3-1272.1 GOPS, and 11.8 FPS under 109.8 KB RAM consumption, which are 3.34x , 1.39x , and 0.62x compared with the state-of-the-art works.