Displaying confidence from imperfect automatic speech recognition for captioning

As the accuracy and latency of Automatic Speech Recognition (ASR) technology improves over time, it may become a viable method for transcribing audio input in real-time for specific situations. Such technology can provide access to spoken language for people who are Deaf or Hard of Hearing (DHH). However, ASR is imperfect and will remain in that state for a while, thus there is a need for users to cope with errors in the output. My research focuses on how to best present captions that make use of the ASR system's word-level confidence. This summary will describe the proposed solution, current state of study, and the planned contribution to the field of HCI and accessibility for DHH individuals.