RGB-D camera based wearable navigation system for the visually impaired.

A wearable RGB-D camera based indoor navigation system for the blind is proposed.Egomotion estimation is performed using both sparse features and dense point clouds.Probabilistic mapping and traversability analysis for path planning is presented.The system stores and reloads maps to expand coverage area of navigation.The system improves mobility performance complementing the white cane. In this paper, a novel wearable RGB-D camera based indoor navigation system for the visually impaired is presented. The system guides the visually impaired user from one location to another location without a prior map or GPS information. Accurate real-time egomotion estimation, mapping, and path planning in the presence of obstacles are essential for such a system. We perform real-time 6-DOF egomotion estimation using sparse visual features, dense point clouds, and the ground plane to reduce drift from a head-mounted RGB-D camera. The system also builds 2D probabilistic occupancy grid map for efficient traversability analysis which is a basis for dynamic path planning and obstacle avoidance. The system can store and reload maps generated by the system while traveling and continually expand the coverage area of navigation. Next, the shortest path between the start location to destination is generated. The system generates a safe and efficient way point based on the traversability analysis result and the shortest path and updates the way point while a user is constantly moving. Appropriate cues are generated and delivered to a tactile feedback system to guide the visually impaired user to the way point. The proposed wearable system prototype is composed of multiple modules including a head-mounted RGB-D camera, standard laptop that runs a navigation software, smart phone user interface, and haptic feedback vest. The proposed system achieves real-time navigation performance at 28.6Hz in average on a laptop, and helps the visually impaired extends the range of their activities and improve the orientation and mobility performance in a cluttered environment. We have evaluated the performance of the proposed system in mapping and localization with blind-folded and the visually impaired subjects. The mobility experiment results show that navigation in indoor environments with the proposed system avoids collisions successfully and improves mobility performance of the user compared to conventional and state-of-the-art mobility aid devices.