Publication: Spontaneous smile intensity estimation by fusing saliency maps and convolutional neural networks
Loading...
Program
KU-Authors
Organization Authors
Co-Authors
Wei, Qinglan
Morency, Louis-Philippe
Sun, Bo
Date
Language
Type
Embargo Status
N/A
Journal Title
Journal ISSN
Volume Title
Alternative Title
Abstract
Smile intensity estimation plays important roles in applications such as affective disorder prediction, life satisfaction prediction, camera technique improvement, etc. In recent studies, many researchers applied only traditional features, such as local binary pattern and local phase quantization (LPQ) to represent smile intensity. To improve the performance of spontaneous smile intensity estimation, we introduce a feature set that combines the saliency map (SM)-based handcrafted feature and non-low-level convolutional neural network (CNN) features. We took advantage of the opponent-color characteristic of SMs and the multiple convolutional level features, which were assumed to be mutually complementary. Experiments were made on the Binghamton-Pittsburgh 4D (BP4D) database and Denver Intensity of Spontaneous Facial Action (DISFA) database. We set the local binary patterns on three orthogonal planes (LBPTOP) method as a baseline, and the experimental results show that the CNN features can better estimate smile intensity. Finally, through the proposed SM-LBPTOP feature fusion with the median- and high-level CNN features, we obtained the best result (52.08% on BP4D, 70.55% on DISFA), demonstrating our hypothesis is reasonable: the SM-based handcrafted feature is a good supplement to CNNs in spontaneous smile intensity estimation.
Source
Publisher
SPIE
Citation
item.page.haspartof
Source
Journal of Electronic Imaging
item.page.ispartofseries
item.page.edition
DOI
10.1117/1.JEI.28.2.023031
item.page.datauri
item.page.link
Rights
N/A
Copyrights Note
Rights and licensing
N/A
