Sign language digits and alphabets recognition by capsule networks

There exist communication barriers between the deaf people and the listeners. Sign language translation is a reasonable and effective way to break these barriers. Recognition of sign language symbols is an essential part of sign language translation. Sign language digits of (0–9) and alphabetic letters of (A–Z) are elementary but important symbols of sign languages of different countries or regions. Capsule networks (CapsNet) are promising alternative to convolutional neural networks (CNN), which take into account of the spatial relationships and orientations of the features of an entity. For sign language digits and alphabets recognition tasks, the proposed SLR-CapsNet architecture achieves a start-of-the-art test accuracy of 99.52% with 100*100 RGB input size and 99.94% with 32*32 RGB input size on Sign Language Digits Dataset and 99.60% with 28*28 Gray-scale input on Sign Language MNIST Dataset. The experimental results also prove that CapsNet has higher generalization and expressiveness capacity on unseen data than CNN dose. Another important finding in our work is that SLR-CapsNet is robust to routing iterations, i.e., its performance will not be affected by various routing iterations.