Publication:
Spontaneous smile intensity estimation by fusing saliency maps and convolutional neural networks

Placeholder

School / College / Institute

Organizational Unit

Program

KU-Authors

KU Authors

Co-Authors

Wei, Qinglan
Morency, Louis-Philippe
Sun, Bo

Publication Date

Language

Embargo Status

Journal Title

Journal ISSN

Volume Title

Alternative Title

Abstract

Smile intensity estimation plays important roles in applications such as affective disorder prediction, life satisfaction prediction, camera technique improvement, etc. In recent studies, many researchers applied only traditional features, such as local binary pattern and local phase quantization (LPQ) to represent smile intensity. To improve the performance of spontaneous smile intensity estimation, we introduce a feature set that combines the saliency map (SM)-based handcrafted feature and non-low-level convolutional neural network (CNN) features. We took advantage of the opponent-color characteristic of SMs and the multiple convolutional level features, which were assumed to be mutually complementary. Experiments were made on the Binghamton-Pittsburgh 4D (BP4D) database and Denver Intensity of Spontaneous Facial Action (DISFA) database. We set the local binary patterns on three orthogonal planes (LBPTOP) method as a baseline, and the experimental results show that the CNN features can better estimate smile intensity. Finally, through the proposed SM-LBPTOP feature fusion with the median- and high-level CNN features, we obtained the best result (52.08% on BP4D, 70.55% on DISFA), demonstrating our hypothesis is reasonable: the SM-based handcrafted feature is a good supplement to CNNs in spontaneous smile intensity estimation. (C) 2019 SPIE and IS&T

Source

Publisher

Spie-Soc Photo-Optical Instrumentation Engineers

Subject

Engineering, Electrical electronic engineering, Optics, Imaging science, Photographic technology

Citation

Has Part

Source

Journal of Electronic Imaging

Book Series Title

Edition

DOI

10.1117/1.JEI.28.2.023031

item.page.datauri

Link

Rights

Copyrights Note

Endorsement

Review

Supplemented By

Referenced By

0

Views

0

Downloads

View PlumX Details