Real-time speech driven gesture animation

Abstract

Gesture and speech co-exist in time with a tight synchrony, and they are planned and shaped by the emotional state and produced together. In our early studies we have developed joint gesture-speech models and proposed algorithms for speech driven gesture animation. These algorithms mainly based on the Viterbi decoders and can not run in realtime. In this paper we are presenting necessary modifications of these algorithms to run in realtime. In the experimental studies we present the conditions that satisfy realtime factor of 1. We as well demonstrate the realtime and non-realtime speech driven gesture animations for subjective evaluations.