<link rel="stylesheet" href="styles.f3b1fba60ec7970c.css">

Publication:
Spontaneous smile intensity estimation by fusing saliency maps and convolutional neural networks

Loading...
Thumbnail Image

Departments

School / College / Institute

Item type:Organizational Unit,

Program

KU-Authors

Organization Authors

Co-Authors

Wei, Qinglan

Morency, Louis-Philippe

Sun, Bo

Date

Language

Embargo Status

N/A

Journal Title

Journal ISSN

Volume Title

Alternative Title

Abstract

Smile intensity estimation plays important roles in applications such as affective disorder prediction, life satisfaction prediction, camera technique improvement, etc. In recent studies, many researchers applied only traditional features, such as local binary pattern and local phase quantization (LPQ) to represent smile intensity. To improve the performance of spontaneous smile intensity estimation, we introduce a feature set that combines the saliency map (SM)-based handcrafted feature and non-low-level convolutional neural network (CNN) features. We took advantage of the opponent-color characteristic of SMs and the multiple convolutional level features, which were assumed to be mutually complementary. Experiments were made on the Binghamton-Pittsburgh 4D (BP4D) database and Denver Intensity of Spontaneous Facial Action (DISFA) database. We set the local binary patterns on three orthogonal planes (LBPTOP) method as a baseline, and the experimental results show that the CNN features can better estimate smile intensity. Finally, through the proposed SM-LBPTOP feature fusion with the median- and high-level CNN features, we obtained the best result (52.08% on BP4D, 70.55% on DISFA), demonstrating our hypothesis is reasonable: the SM-based handcrafted feature is a good supplement to CNNs in spontaneous smile intensity estimation.

Source

Publisher

SPIE

Citation

item.page.haspartof

Source

Journal of Electronic Imaging

item.page.ispartofseries

item.page.edition

DOI

10.1117/1.JEI.28.2.023031

item.page.datauri

item.page.link

Rights

N/A

Copyrights Note

Rights and licensing

N/A

Endorsement

Review

Supplemented By

Referenced By

Related Patent

Related Goal

Google Scholar
Scholar'da Ara ↗
Altmetric
Dimensions
PlumX Metrikleri
BIP! Indicators