<link rel="stylesheet" href="styles.f3b1fba60ec7970c.css">

Publication:
An audio-driven dancing avatar

Loading...
Thumbnail Image

School / College / Institute

Item type:Organizational Unit,
Item type:Organizational Unit,
Item type:Organizational Unit,

Program

Organization Authors

Co-Authors

Balcı, Koray

Kızoğlu, İdil

Akarun, Lale

Canton-Ferrer, Cristian

Tilmanne, Joelle

Bozkurt, Elif

Erdem, A. Tanju

Date

Language

Embargo Status

N/A

Journal Title

Journal ISSN

Volume Title

Alternative Title

Abstract

We present a framework for training and synthesis of an audio-driven dancing avatar. The avatar is trained for a given musical genre using the multicamera video recordings of a dance performance. The video is analyzed to capture the time-varying posture of the dancer's body whereas the musical audio signal is processed to extract the beat information. We consider two different marker-based schemes for the motion capture problem. The first scheme uses 3D joint positions to represent the body motion whereas the second uses joint angles. Body movements of the dancer are characterized by a set of recurring semantic motion patterns, i.e., dance figures. Each dance figure is modeled in a supervised manner with a set of HMM (Hidden Markov Model) structures and the associated beat frequency. In the synthesis phase, an audio signal of unknown musical type is first classified, within a time interval, into one of the genres that have been learnt in the analysis phase, based on mel frequency cepstral coefficients (MFCC). The motion parameters of the corresponding dance figures are then synthesized via the trained HMM structures in synchrony with the audio signal based on the estimated tempo information. Finally, the generated motion parameters, either the joint angles or the 3D joint positions of the body, are animated along with the musical audio using two different animation tools that we have developed. Experimental results demonstrate the effectiveness of the proposed framework.

Source

Publisher

Springer

Citation

item.page.haspartof

Source

Journal on Multimodal User Interfaces

item.page.ispartofseries

item.page.edition

DOI

10.1007/s12193-008-0009-x

item.page.datauri

item.page.link

Rights

N/A

Copyrights Note

Rights and licensing

N/A

Endorsement

Review

Supplemented By

Referenced By

Related Patent

Related Goal

Google Scholar
Scholar'da Ara ↗
Altmetric
Dimensions
PlumX Metrikleri
BIP! Indicators