Publication:
Audio-facial laughter detection in naturalistic dyadic conversations

Placeholder

Organizational Units

Program

KU Authors

Co-Authors

N/A

Advisor

Publication Date

2017

Language

English

Type

Journal Article

Journal Title

Journal ISSN

Volume Title

Abstract

We address the problem of continuous laughter detection over audio-facial input streams obtained from naturalistic dyadic conversations. We first present meticulous annotation of laughters, cross-talks and environmental noise in an audio-facial database with explicit 3D facial mocap data. Using this annotated database, we rigorously investigate the utility of facial information, head movement and audio features for laughter detection. We identify a set of discriminative features using mutual information-based criteria, and show how they can be used with classifiers based on support vector machines (SVMs) and time delay neural networks (TDNNs). Informed by the analysis of the individual modalities, we propose a multimodal fusion setup for laughter detection using different classifier-feature combinations. We also effectively incorporate bagging into our classification pipeline to address the class imbalance problem caused by the scarcity of positive laughter instances. Our results indicate that a combination of TDNNs and SVMs lead to superior detection performance, and bagging effectively addresses data imbalance. Our experiments show that our multimodal approach supported by bagging compares favorably to the state of the art in presence of detrimental factors such as cross-talk, environmental noise, and data imbalance.

Description

Source:

Ieee Transactions On Affective Computing

Publisher:

Ieee-Inst Electrical Electronics Engineers Inc

Keywords:

Subject

Computer science, Artificial intelligence, Computer science, Cybernetics

Citation

Endorsement

Review

Supplemented By

Referenced By

Copy Rights Note

0

Views

0

Downloads

View PlumX Details