Enhancing Gesture Recognition in Low-Light Conditions Using Laser-Induced Graphene Flexible Strain Sensors

Gesture recognition using machine learning methods is valuable in the development of healthcare and human-computer interaction, and typically relies on images or video. To improve recognition accuracy, these visual data can be combined with data from other sensors, but this approach, known as data fusion, is limited by the quality of sensor data and incompatible datasets. Here, we propose a flexible strain sensor based on laser-induced graphene technology, which is easy to manufacture and has superior performance. The sensor is attached to the finger and effectively captures finger bending information as an additional gesture information channel, constructing a bimodal gesture interaction interface for strain and vision. In particular, a deep convolutional neural network based on machine learning is used to acquire features in monocular camera image data. The data fusion method of feature layer fusion is selected to construct a bimodal gesture recognition system based on image data and strain sensor data. Experimental tests show that the strain and vision bimodal gesture interaction interface developed based on this fusion recognition model can still achieve more than 90% real-time gesture recognition accuracy under an illumination level of 20 lux, which effectively solves the problem of the traditional monocular vision’s gesture interaction interface with a significant decrease in recognition accuracy under dark light conditions, and also further expands the application of LIG flexible sensor devices in human-computer interaction system.