Publication:
Lip feature extraction based on audio-visual correlation

Placeholder

Program

School / College / Institute

College of Engineering
GRADUATE SCHOOL OF SCIENCES AND ENGINEERING

KU Authors

Co-Authors

Publication Date

Language

Embargo Status

Journal Title

Journal ISSN

Volume Title

Alternative Title

Abstract

In this paper, the lip feature that has the highest correlation with audio features is investigated. Audio features are selected as Mel Frequency Cepstral Coefficients (MFCC) of the audio signal. Three different lip features are considered for the visual lip information, where these features are 2D DCT coefficients of the intensity based image and the optical flow vectors within the lip region, and the distances between pre-defined points on the lip contour which carries the lip shape information. In this study, we present two techniques based on class conditional probability analysis and canonical correlation analysis to estimate and compare the correlations between audio feature and each lip feature. The lip feature, which has the highest correlation to audio features, is identified among the above lip features. Isolation of lip features, which are highly correlated with audio signal, can be used for audio-visual speech recognition, audio-visual lip synchronization and estimation of lip shapes using audio signal for visual synthesis.

Source

Publisher

European Association for Signal Processing

Subject

Engineering

Citation

Has Part

Source

13th European Signal Processing Conference, EUSIPCO 2005

Book Series Title

Edition

DOI

item.page.datauri

Rights

Rights URL (CC Link)

Copyrights Note

Endorsement

Review

Supplemented By

Referenced By

0

Views

0

Downloads