Joint Segmentation and Sub-pixel Localization in Structured Light Laryngoscopy

In recent years, phoniatric diagnostics has seen a surge of interest in structured light-based high-speed video endoscopy, as it enables the observation of oscillating human vocal folds in vertical direction. However, structured light laryngoscopy suffers from practical problems: specular reflections interfere with the projected pattern, mucosal tissue dilates the pattern, and lastly the algorithms need to deal with huge amounts of data generated by a high-speed video camera. To address these issues, we propose a neural approach for the joint semantic segmentation and keypoint detection in structured light high-speed video endoscopy that improves the robustness, accuracy, and performance of current human vocal fold reconstruction pipelines. Major contributions are the reformulation of one channel of a semantic segmentation approach as a single-channel heatmap regression problem, and the prediction of sub-pixel accurate 2D point locations through weighted least squares in a fully-differentiable manner with negligible computational cost. Lastly, we expand the publicly available Human Laser Endoscopic dataset to also include segmentations of the human vocal folds itself. The source code and dataset are available at: github.com/Henningson/SSSLsquared