Automatic vocal tract landmark tracking in rtMRI using fully convolutional networks and Kalman filter

Abstract

Vocal tract (VT) contour detection in real time MRI is a pre-stage to many speech production related applications such as articulatory analysis and synthesis. In this work, we present an algorithm for robust detection of keypoints on the vocal tract in rtMRI sequences using fully convolutional networks (FCN) via a heatmap regression approach. We as well introduce a spatio-temporal stabilization scheme based on a combination of Principal Component Analysis (PCA) and Kalman filter (KF) to extract stable landmarks in space and time. The proposed VT landmark detection algorithm generalizes well across subjects and demonstrates significant improvement over the state of the art baselines, in terms of spatial and temporal errors.