Vision sensory substitution to aid the blind in reading and object recognition

Vision Sensory Substitution is a non-invasive rehabilitating procedure for the blind, which provides visual information to them via any of their functioning senses. In this paper, the authors propose a more engaging, interactive and integrated technology which will enable the visually impaired `to read' the text and `to see' their surroundings with the aid of computer vision. For reading of the text, Optical Character Recognition engine is utilized so that the blind can listen to the text which was intended to be read. For seeing objects and captioning them, features are extracted using Histogram of Oriented Gradients and the classification and labelling of the object via its features is done using K-Nearest Neighbor classifier. Results show that a high accuracies of 92% and 88% have been achieved for text extraction and caption generation respectively. The text and captions (also in text form) are finally converted into audio signals using text to speech converters.