Disease Detection from Audio-Visual Signals: Recent Advancements and Challenges

Technology is advancing in the area of disease detection from audio-visual signals. Ambient cameras, which are ubiquitous nowadays, capture audio and video signals from individuals. These signals can be used to detect an individual’s physiological and behavioural abnormalities which may play a key role in assessing his current mental and physiological condition. The signals can also aid in the diagnosis of many disorders, especially psychiatric and neurological. The problem of disease detection from audio-visual signals is relatively new and has the potential to generate exciting scientific contributions. However, a number of challenges lay ahead. The ability to identify an individual’s mental and emotional state from audio-visual footage can be a significant step towards tackling a number of difficult societal problems. For decades, researchers have developed artificial intelligence, machine learning, computer vision, and image and signal processing algorithms to learn useful features, extract contextual information, and generate high-level knowledge from audio-visual footage. These algorithms are useful in the detection of abnormalities, and classification of diseases and disorders. These methods can apply to any individual whose changes in physical appearance and vocal/physical sounds due to environmental stimuli can be captured by cameras already available everywhere. The audio-visual footage allows monitoring individuals in specific and rare situations that can provide crucial health information leading to urgent attention and care enabling a quick recovery.  This paper summarizes the algorithmic approaches, including preprocessing techniques, for different kinds of audio-visual signals, and their applications and effectiveness in disease/disorder detection. It highlights and evaluates the widely-used state-of-the-art machine learning approaches, and identifies the scope for new methods and algorithms. The paper also summarizes the challenges in implementing such technologies in the real world.