Trainable Power-of-2 Scale Factors for Hardware-friendly Network Quantization

Quantization can efficiently reduce the bitwidth of parameters in neural networks and accelerate both inference and data transmission, which is essential for deploying networks with a large parameter scale on resource-limited edge devices. However, most of the existing quantization methods use non-linear function (e.g., tanh, non-linear polynomial function), it is not only difficult to implement on hardware but also occupy plenty of computing resources. In addition, previous quantization methods seldom consider the hardware implementation of operators, which only simulate the quantization accuracy at the algorithm level. To address these problems, we propose novel \emph{Trainable Power-of-2 Scale Factors Quantization} (TPSQ) to combine power-of-2 quantization scale factor and trainable clamp interval, to benefit from the advantages of both. Several experiments over current mainstream vision tasks show that the performance of TPSQ surpasses most of the previous quantization methods, these prove the effectiveness of TPSQ. Finally, we implemented an FPGA accelerator for object detection to demonstrate the hardware friendliness of TPSQ. The experimental results show that the peak performance of our system is 300 GOP/s under 300 MHz working frequency, which is 72 times higher than the implementation on an ARM-A9 processor.