A High Speed and Resource Efficient Approximation of Softmax Loss Function

This work focuses primarily on the successful design and implementation of a high speed and a resource efficient approximation of Softmax loss function. The implementation explores system generator tool together with Simulink as a viable alternative for rapid prototyping, evaluation, and deployment of Field Programmable Gate Array (FPGA) based solutions. Linear look-up table (LUT) based interpolation is used for approximation of non-linear functions such as exponential as well as the implementations of dividers. They are usually very costly when designed through conventional approaches. After rigorous analysis, a dynamic quantization scheme is developed which offers the best compromise between accuracy and speed of execution. These results are then further expanded to extract rules for the implementation of softmax approximators of varying sizes. The results show that for an architecture with the OEM claimed maximum frequency of 600 MHz the proposed system achieves a synthesis frequency of 400 MHz. These results show that design using Xilinx system generator provides designer access to a vast variety of data visualization and processing tools that come packed with Matlab Simulink without significant performance degradation.