Improving throat microphone speech recognition by joint analysis of throat and acoustic microphone recordings

Abstract

We present a new framework for joint analysis of throat and acoustic microphone (TaM) recordings to improve throat microphone only speech recognition. the proposed analysis framework aims to learn joint sub-phone patterns of throat and acoustic microphone recordings through a parallel branch HMM structure. the joint sub-phone patterns define temporally correlated neighborhoods, in which a linear prediction filter estimates a spectrally rich acoustic feature vector from throat feature vectors. Multimodal speech recognition with throat and throat-driven acoustic features significantly improves throat-only speech recognition performance. Experimental evaluations on a parallel TaM database yield benchmark phoneme recognition rates for throat-only and multimodal TaM speech recognition systems as 46.81% and 60.69%, respectively. the proposed throat-driven multimodal speech recognition system improves phoneme recognition rate to 52.58%, A significant relative improvement with respect to the throat-only speech recognition benchmark system.