FAST: FPGA Acceleration of Neural Networks Training

Training state-of-the-art ANNs is computationally and memory intensive. Thus, implementing the training on embedded devices with limited resources is challenging. In order to address this challenge, we propose FAST, a low-precision method to implement and optimize ANN training on FPGA. FAST first addresses the challenge of implementing the non-polynomial sigmoid activation function by presenting a solution using PNLA methods. Then, it introduces Hardware optimized PReLU (HOPE) activation function, which is specifically devised to reduce the required resources and increase the accuracy of computations on FPGA. We evaluated FAST against the software implementations of ANNs, using training tasks available in the MNIST benchmark. The results show that FAST improves the training speed by 8.6$\times$ and reduces the required memory size by orders of magnitude. It is worthwhile to mention that the method imposes almost no degradation in training accuracy.