Hand Gesture Recognition using a Convolutional Neural Network

In this paper we present an approach towards hand gesture recognition that uses a Convolutional Neural Network (CNN), which is trained on Discrete Fourier Transform (DFT) images that result from raw sensor readings. More specifically, we use the Kinect RGB and depth camera and we capture the 3D positions of a set of skeletal joints. From each joint we create a signal for each 3D coordinate and we concatenate those signals to create an image, the DFT of which is used to describe the gesture. We evaluate our approach using a dataset of hand gestures involving either one or both hands simultaneously and compare the proposed approach to another that uses hand-crafted features.